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ABSTRACT 


This study was designed to investigate the relationship between participation in federally 
mandated college and career-preparatory programs — known as programs of study (POS) — and 
high school achievement outcomes. POS are an organized approach to college and career 
readiness that offer an aligned sequence of courses spanning secondary and postsecondary 
education, blending standards-based academic and technical content, allowing students to earn 
postsecondary credit while in high school, and leading to an industry-recognized credential or 
certificate at the postsecondary level or an associate or baccalaureate degree. 

The sample includes 6,638 students from three urban districts in three different states. This study 
employed a multi-method, longitudinal, quasi-experimental design. Qualitative measures 
included adherence to the legislatively mandated components of POS. In the quantitative portion 
of the study, we employed two different statistical approaches to the data in each district. First, 
we estimated the effects of enrolling in POS and the number of career and technical education 
(CTE) credits earned on GPA and graduation using an instrumental variable approach. In 
addition, we also addressed specific policy questions about completing a POS compared to other 
high school trajectories through posthoc multiple regression analyses. 

The outcomes show that in the first district, enrollment in POS schools caused more students to 
graduate by increasing the number of CTE credits they earned. In all three districts, earning more 
CTE credits was associated with graduation, although the results for the other two districts did 
not support the type of causal inferences we were able to draw from the first district’s outcomes. 
We also found that POS students earned significantly more STEM or AP credits than comparison 
students, depending on the implementation context. POS students outperfonned other students 
on technical measures at little to no cost to overall academic achievement. 

Across all districts, participation in programs associated with accruing college credits in high 
school (e.g., dual enrollment) was low, as was the incidence of earning an industry-recognized 
credential, two key elements of POS. We lacked the research funding to examine whether POS 
led to postsecondary degrees; however, senior exit survey results indicated that similar numbers 
of intervention and comparison students planned to attend a four-year college full time. This 
suggests that POS can be offered to high school students with no hann to their college 
aspirations or preparation. In addition, significantly more POS students indicated that their 
college studies would be related to their high school program, suggesting that students who 
enroll in a POS often continue their education in the same program area, and reap the benefits of 
having begun that preparation while still in high school. 

Our primary recommendation is for districts to seek ways to increase the number of CTE credits 
earned by high school students, because in all three districts, earning more CTE credits was 
associated with graduation. Other recommendations include re-examining dual enrollment 
programs so that more students participate, encouraging more people from industry backgrounds 
to become high school teachers, and improving data collection so that the efficacy of 
interventions may be better evaluated. All of these recommendations should improve POS, 
secondary school outcomes, and preparation of our nation’s youth for postsecondary education 
and careers. 
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EXECUTIVE SUMMARY 


Introduction 

This study was designed to investigate the relationship between participation in federally 
mandated college and career-preparatory programs — known as programs of study (POS) — and 
high school achievement outcomes. POS are an organized approach to college and career 
readiness that offer an aligned sequence of courses spanning secondary and postsecondary 
education, blending rigorous academic and technical content, allowing students to earn 
postsecondary credit while in high school, and leading to an industry-recognized credential or 
certificate at the postsecondary level or an associate or baccalaureate degree. 

This study was proposed in the context of the reauthorization of the Carl D. Perkins legislation 
(2006), which funds CTE nationwide. The reauthorization, known as Perkins IV, increased 
program accountability in the areas of academic achievement, technical skills achievement, and 
alignment with postsecondary technical education — in the fonn of POS. In the Perkins IV 
legislation, POS have four components. They: 

• must incorporate secondary education and postsecondary education elements; 

• must include coherent and rigorous content aligned with challenging academic standards 
and relevant career and technical content in a coordinated, non-duplicative progression of 
courses that align secondary education with postsecondary education to adequately 
prepare students to succeed in postsecondary education; 

• may include the opportunity for secondary education students to participate in dual or 
concurrent enrollment programs or other ways to acquire postsecondary education 
credits; and 

• must lead to an industry-recognized credential or certificate at the postsecondary level, or 
an associate or baccalaureate degree. 

The primary research question guiding this study has been, does participation in the high school 
portion of a POS correlate with improved student outcomes? We used common high school 
outcomes, and we also incorporated a qualitative design to help explain the contexts in which the 
outcomes were achieved. 

In this report, we situate the study within current national educational and economic contexts. As 
college and career readiness has become the federal focus of public education reform, states and 
districts have sought to implement programs like POS that address readiness gaps and prepare 
students for further education and training. Career-focused education that blends standards- 
based, integrated academics with hands-on, challenging, career-ready skills, enriched by 
educational partnerships with industry, are designed to help mitigate the effects of poverty and 
low parental educational attainment and positively influence students’ postsecondary aspirations. 

Method 

This study employed a multi-method, longitudinal, quasi-experimental design to investigate the 
relationship between POS participation and the high school outcomes described in the Perkins 



legislation for assessing the effectiveness of POS. The study consists of three strands employing 
slightly different research designs and separate analyses, one for each participating district. We 
employed this tripartite design because (a) the districts operate in very different contexts, with 
quite different state policy contexts; and (b) due to contextual differences, the means of creating 
samples differed across sites. Further, one district began participating in the study well after the 
others, meaning that some data could not be collected there (e.g., classroom observations and 
interviews for students’ first three years of high school). 

The sample includes 6,638 students from three urban districts in three different states. Minority 
students were in the majority in these districts. Student participants included lottery applicants in 
the two districts that held lotteries for entry into specialized high school programs such as POS, 
and in the third, we included students enrolled in the POS high school and a district-created 
propensity score -matched comparison group. Despite the presence of lotteries in two of the three 
districts, students were not randomly assigned to the POS, so an experimental strand was ruled 
out. 

Our outcome measures are found in district systems data and are of practical interest to schools. 
The intent of Perkins IV is to improve the achievement of CTE students in relation to state 
academic standards and to ease their transition to postsecondary education. We operationalized 
these POS elements into outcomes that capture the skills and content that POS were designed to 
address. These include: 

(a) Academic achievement as measured by cumulative grade point average (GPA), STEM 
credits earned (i.e., credits in science and in higher math, defined as math above Algebra 
II), and AP credits earned; 

(b) Technical skills achievement, measured by CTE GPA; 

(c) Graduation rates, and 

(d) Transition measures such as post -high school plans and completion of coursework 
associated with college credits (i.e., dual enrollment) 

Qualitative measures included adherence to the components of POS as laid out by Perkins IV 
and a policy guidance framework subsequently released by the U.S. Department of Education’s 
Office of Adult and Vocational Education, which oversees Perkins implementation. 

In the quantitative portion of the study, we employed two different statistical approaches to the 
data in each district, in order to answer important policy questions about POS. First, we 
estimated the effects of enrolling in POS and number of CTE credits earned on GPA and 
graduation using an instrumental variable approach. In addition to that main analysis, we also 
addressed specific policy questions about completing a POS compared to other high school 
trajectories through posthoc multiple regression analyses. 

The qualitative portion of the study consisted of data collected on annual site visits, including 
interviews and classroom observations. Interviews were transcribed and observation forms were 
completed. These data were coded, which involved marking the instances of concepts or topics 
relevant to the study as they appeared in the interview or observation data. Codes were entered 
into a qualitative data analysis software package, the corpus of which could then be queried to 
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yield reports rich with information on each code from every type of interviewee and observed 
class. These reports allowed us to group the data into more specific topical and descriptive 
categories, generating an understanding of POS in these contexts that helped explain the diverse 
outcomes across districts. 

Other data included a senior exit survey administered to elicit student post-high school plans. 

Outcomes 

Specific outcomes are presented by district, but some findings spanned districts: Across all 
districts, participation in programs associated with accruing college credits in high school (e.g., 
dual enrollment) was low. The analytical model indicated a weak fit to the data and suggested 
that important factors in dual enrollment participation were not being captured. Across the two 
districts that held a senior exit survey, results revealed very similar post-high school plans for 
both students who enrolled in POS and the comparison group: the majority of both groups 
planned to attend a four-year college or university full time, and had already been accepted to the 
school of their choice. Intervention students were more likely to choose a college major related 
to their high school program area than were those comparison students who were in a specific 
program in high school (i.e., performing arts). Finally, for each district, a qualitative analysis of 
POS implementation is presented, based on the elements of the law mandating POS and the 
subsequent policy guidance framework. 


West District 

West District is an urban district that experienced a large population increase prior to the recent 
economic recession. To respond to this growth, new high schools were built and integrated into 
the district’s school choice program. The impetus behind school choice was to improve student 
achievement, promote diversity, and create an awareness of career opportunities relative to POS 
and magnet themes. The school cultures at the three participating POS high schools fostered 
environments where students could both focus on their chosen POS and excel academically. 

We tested an instrumental variable model of the effects of POS enrollment and CTE credit 
earning on GPA and graduation. Our results indicate that enrollment in POS high schools 
improved students’ probability of graduation by 1 1.3% and that increased CTE credit earning 
fully explained this effect. This analysis also showed non-significant effects of POS enrollment 
and CTE credit earning on overall high school GPA. These findings suggest that enrollment in 
West’s POS schools benefited students in terms of retention, at no cost to their achievement. 

In West District, 49.5% of the intervention group completed a POS. We examined policy- 
relevant questions using multiple regression models to compare POS completers to CTE 
concentrators (a Perkins-derived designation for students earning a state-determined number of 
CTE credits in a program area), and to the rest of the sample. We found that compared to the rest 
of the sample, POS completers were more likely to 1) have a higher overall GPA, 2) have a 
higher CTE GPA, and 3) earn more STEM credits, but completing a POS did not affect the 
likelihood of earning AP credits. In the comparison with CTE concentrators, POS completers 
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were more likely to 1) have a higher overall GPA, and 2) earn more STEM credits, but 
completing a POS did not affect the likelihood of earning STEM or AP credits. 

East District 

Like many urban school districts, East District developed multiple magnet and choice programs 
across grade spans. At the high school level, a number of schools or programs within schools 
offered a range of magnets available to students. Blue Academy, the wall-to-wall magnet POS 
high school that participated in this study, served students from all over East District. Although 
Blue held very high expectations for its students, teachers, students, and administrators described 
a supportive, caring culture where barriers to success were identified and addressed. 

Results from the instrumental variable model at East District were not interpretable because it 
failed to generate substantial exogenous variation in number of CTE credits earned, so we tested 
a conventional mediator model. We found that although POS enrollment did not impact CTE 
credit earning, each additional CTE credit earned corresponded to a 9% improvement in the 
probability of graduation and a .15 unit improvement in GPA. These results suggest that earning 
CTE credits benefited all students — both intervention and comparison — in tenns of retention and 
also achievement. 

In East District, 50.0% of the intervention group completed a POS. In assessing the effect of 
completing a POS on high school achievement, we found that POS completers were much more 
likely than the rest of the sample to have earned more AP credits, but completing a POS at Blue 
did not affect the likelihood of earning higher GPAs overall or in CTE, or the likelihood of 
earning STEM credits. 

Because there was only one intervention school in East District, we compared POS completer 
outcomes to those of CTE concentrators in Blue Academy (i.e., Blue POS students who failed to 
complete their POS but qualified for CTE concentrator status) as opposed to all CTE 
concentrators in the sample. We did this in order to test whether outcomes were attributable to 
completing a POS or if they had more to do with Blue Academy as a school — its reputation, 
culture, and faculty. Compared to Blue concentrators, completing a POS did not affect the 
likelihood of earning a higher overall or CTE GPA, or of earning more STEM or AP credits, 
suggesting that Blue Academy as a school may be more responsible for the outcomes as opposed 
to completing a POS. 


South District 

South District is located in a state that strongly encouraged high schools to develop career 
academies as a way to engage students and lead them to high-demand, high-wage careers. This 
state mandated articulation agreements and dual enrollment opportunities, and included industry- 
recognized credentials as part of a college-and-career-ready accountability program. Rather than 
create POS high schools, South District ensured that all students had the opportunity to 
participate by developing POS in every high school. Some of these POS were accessible through 
the district’s lottery system, and others were available for students attending their zoned high 
schools. There were no systematic differences between these programs. 
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Like in East District, the instrumental variable model in South also failed to generate substantial 
exogenous variation in number of CTE credits earned, precluding us from interpreting the 
results. We tested a conventional mediator model and found that (1) students who enrolled in 
POS were 9% more likely to graduate than students who enrolled in other programs, (2) students 
who enrolled in POS had slightly better GPAs than students enrolled elsewhere, and (3) these 
effects were attributable to increased earning of CTE credits. These results suggest that 
enrollment in POS may have benefited students in terms of retention and also achievement. 

In South District, 58.0% of the intervention group completed a POS. We assessed the effect of 
completing a POS on high school achievement, finding that compared to the rest of the sample, 
South POS completers were significantly less likely to earn more 1) STEM credits or 2) AP 
credits, but they were more likely to have a higher CTE GPA. The results comparing POS 
completers with CTE concentrators were similar: POS completers were significantly less likely 
to earn more 1) STEM credits or 2) AP credits, but they were more likely to have a higher CTE 
GPA. 

Unlike the other two districts participating in this study, no senior exit survey was conducted in 
South District. However, we were able to collect some industry-recognized credential data from 
South, which was not available at the other districts. The data were not complete, but among the 
students known to have sat for a credentialing exam, nearly 94% passed and earned the 
credential. Over three quarters of those credentials were earned by POS completers. 

Telling the Story of POS: Cross-District Synthesis 

We integrated the findings from the mixed methods of this study to synthesize what we learned 
about the impact of POS on student achievement. 

Context for the Achievement Outcomes: West District 

Regular CTE programs in West District did not offer the same rigor as POS — this is evident in 
both GPA and STEM credits outcomes. POS completers had significantly, substantively higher 
GPAs than both comparison groups. That, coupled with the higher graduation rate among 
intervention students, suggests that POS motivated those students to achieve and finish high 
school. We posit that in addition to completing more CTE credits, the additional elements of 
POS helped foster engagement and achievement. Students in POS knew that their programs had 
a built-in next step at the local community college or university. School cultures at the POS high 
schools fostered individual and career development, opportunities to lead, positive attitudes 
about achievement and greater commitment to careers. Students knew that their curriculum was 
at a higher level than at other schools — that their preparation was college-level in many cases, 
and that it could lead to advanced status in college. 
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Context for the Achievement Outcomes: East District 

The entire Blue Academy staff promoted college attendance as one of the primary student 
outcomes they sought. The principal claimed that he pushed students to take AP courses, 
especially those that were in any way aligned with the POS theme. We did not observe this 
overriding goal of college attendance at the comparison schools. This may account for the 
significant and substantively important positive outcome of earning AP credits at Blue compared 
to the rest of the sample. The rest of the findings were neither significant nor did they generate 
substantive effect sizes, indicating that POS may be offered in districts like East at no cost to 
other academic achievement measures. 

Context for the Achievement Outcomes: South District 

Intervention students in South District had slightly better GPAs and were more likely to graduate 
than the comparison group, although causal inference remains tentative. CTE concentrators and 
the rest of the sample earned significantly higher numbers of STEM and AP credits than POS 
completers did. It is possible that the additional CTE credit and other requirements of POS at 
South District (e.g., internships, industry credentials) took enough time away from POS 
completers that they were unable to pursue as many advanced academic courses. We know from 
the qualitative data that many comparison group students were in college-preparatory programs 
that required advanced academic course-taking, which could help explain the lack of positive 
outcomes for POS completers on those measures. South POS completers had significantly higher 
CTE GPAs than CTE concentrators and the rest of the sample, suggesting that they had focused 
on their POS area over advanced academic course-taking. 

Across Contexts 

In sum, the findings from the three districts indicate that taking more CTE credits may boost 
GPA, the probability of graduation, and some achievement measures, and that these results come 
at little to no cost to overall academic achievement, based on the posthoc findings. 

With respect to dual enrollment, despite the promotion of these credit-based transfer mechanisms 
within POS, completing a POS was not associated with accruing more college credits. In fact, 
when there was a significant difference between POS completers and the other groups, as there 
was in West and East Districts, the difference favored the rest of the sample, not POS 
completers. The models we ran on college credits accrued were not as well-aligned with the data 
as the other analyses, so this may have affected the results. We know from the qualitative data 
that all students, not just POS students, were encouraged to take courses that offered the 
opportunity to earn college credits. It could be that non-POS students had more time in their 
schedules. Or, as the qualitative data suggested, POS students in these districts saw themselves 
as bound for four- year colleges and universities, and the community college credits that were 
being offered did not seem as useful or desirable to them as other college-preparatory approaches 
like AP and other advanced course-taking. 

In terms of industry-recognized credentials, no usable data were collected in West or East 
Districts. In South District, the data we obtained were known to be incomplete, but provided a 
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rough picture of credential earning by POS students. About 16% of our posthoc sample took an 
exam for an industry credential. Of those students who took an exam and earned an industry 
credential, three quarters were POS completers. 

POS are by definition secondary-postsecondary programs, designed to lead to associate and 
baccalaureate degrees. Although we lacked research funding to follow these Class of 2012 
cohorts immediately into postsecondary education and careers, future research on this sample 
would provide important infonnation on the postsecondary half of POS. In spite of this 
limitation, senior exit survey results indicated that similar numbers of intervention and 
comparison students planned to attend a four-year college full time in both West and East 
Districts (South did not participate in the senior exit survey). This suggests that POS can be 
offered to high school students with no harm to their college aspirations or preparation. 

Notably, survey data showed that significantly more POS students in both West and East 
Districts indicated that their college studies would be related to their high school programs. This 
suggests that students who enroll in a POS may continue their education in the same program 
area and reap the benefits of beginning that preparation in high school. Qualitative data showed 
this to be a purposeful goal of POS: Teachers and administrators in both districts spoke of 
putting students on a fast track to college by (a) getting the foundational and pre-requisite 
courses out of the way while still in high school, and (b) allowing students to earn industry 
credentials that would help them work in their fields while continuing their studies. 

Recommendations 

Our primary recommendation is for districts to seek ways to increase the number of CTE credits 
earned by high school students. Our findings for West District suggested that enrollment in POS 
schools caused more students to graduate by increasing the number of CTE credits they earned. 
In all three districts, earning more CTE credits was associated with graduation. In contexts 
similar to West District, increasing opportunities for students to enroll in POS could improve 
graduation rates. In districts where POS enrollment is unlikely to change CTE credit earning, 
using other means to increase the number of CTE credits earned by secondary students could 
improve graduation rates. 

We conclude that CTE credits need not be presented in a POS sequence to have the desired 
impact on graduation. However, posthoc analyses showed that in most cases, CTE offered in 
POS sequences may have had a more beneficial impact on other important student outcomes, 
more than simply taking CTE courses or completing a CTE concentration. Further, qualitative 
data showed that most students, teachers, and administrators were positively inclined toward 
POS. They believed that POS provided students with experiences and opportunities that 
encouraged engagement and prepared them for their next steps after graduation. Administrators 
and policymakers should consider designing programs around the elements of POS that make the 
most sense for their contexts, then work to improve practice, learning, and support systems (e.g., 
data collection methods) to provide evidence of student outcomes. 

Other recommendations include re-examining dual enrollment programs so that more students 
participate, encouraging more people from industry to become teachers, assigning guidance 
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counselors by high school program instead of student last name, and improving data collection 
so that the efficacy of interventions may be better evaluated. All of these recommendations 
should improve POS, secondary school outcomes, and preparation of our nation’s youth for 
postsecondary education and careers. 

This study confirms that although high-quality CTE programs in the form of POS are not easy, 
cheap, or capable of solving all educational problems, they can be implemented well and yield 
positive results, as the three districts that participated in this study demonstrated. However, this 
study remains an incomplete examination of POS because the postsecondary portion was not 
able to be incorporated here. Future work must include the complete intervention in order to 
arrive at a comprehensive understanding of the impact of POS on student secondary, 
postsecondary, and career outcomes. 



CHAPTER ONE: Rigorous Tests of Student Outcomes in CTE Programs of Study 

Introduction 

This study was designed to investigate the relationship between participation in federally 
mandated college and career-preparatory high school programs — known as programs of study 
(POS) — and high school achievement outcomes. POS are an organized approach to college and 
career readiness that offer an aligned sequence of courses spanning secondary and postsecondary 
education, blending rigorous academic and technical content, allowing students to earn 
postsecondary credit while in high school, and leading to an industry-recognized credential or 
certificate at the postsecondary level or an associate or baccalaureate degree. 

The primary research question guiding this study has been, does participation in the high school 
portion of a college and career-preparatory program correlate with improved student outcomes? 
We use common high school outcomes, and we also incorporated a qualitative design to help 
explain the contexts in which the outcomes were achieved. 

In this report, we first situate the study within current national educational and economic 
contexts. Next, we describe the legislative components of POS, noting that POS were designed 
to be secondary-postsecondary programs, not just an improvement on high school career and 
technical education (CTE) programs. We then review the research base on POS and several of its 
precursors — tech prep, dual enrollment programs, and career academies — on which POS were in 
part based. In fact, states and districts often implement POS by building upon these efforts. In the 
chapters that follow, we present the study’s research questions, study design, and methods, and 
then report the outcomes for each participating district. We use a cross-case analysis to discuss 
these outcomes, and provide recommendations for policy and practice. 

Purpose and Significance of the Study 

This study was proposed in the context of the reauthorization of the Carl D. Perkins legislation 
(2006), which funds CTE nationwide. The reauthorization, known as Perkins IV, modified 
existing practice by increasing program accountability in the areas of academic achievement, 
technical skills achievement, and alignment with postsecondary technical education — in the form 
of POS. 


Description of the Intervention 

According to the Perkins legislation, there are several components to CTE POS, which: 

(i) must incorporate secondary education and postsecondary education elements; 

(ii) must include coherent and rigorous content aligned with challenging academic 
standards and relevant career and technical content in a coordinated, nonduplicative 
progression of courses that align secondary education with postsecondary education to 
adequately prepare students to succeed in postsecondary education; 
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(iii) may include the opportunity for secondary education students to participate in dual 
or concurrent enrollment programs or other ways to acquire postsecondary education 
credits; and 

(iv) must lead to an industry-recognized credential or certificate at the postsecondary 
level, or an associate or baccalaureate degree (Perkins IV, Section 122[c] [ 1 ] [A]). 

Because these were the legal requirements of POS, they were among the criteria we used to 
select our sites. Other criteria included the districts’ ability to support our study design (see 
Method, below). Comparison group students from these districts had CTE programs at the 
comprehensive high schools they attended, and some students availed themselves of these 
programs. Others took academic and elective courses that are typical of the range of such courses 
offered in contemporary American high schools. 

POS differ from comparison school CTE programs in the ways enumerated by the legislation. 
First, POS are characterized by non-duplicative sequencing of secondary and postsecondary 
curriculum, in order to provide seamless transitions. This course sequence at the high school 
level is a key characteristic of POS — the comparison schools have CTE programs but sequenced 
course-taking within those programs is not required. Second, POS need to foster standards-based 
academic course-taking as well as technically rigorous CTE courses as a way to prepare students 
for postsecondary education. Not all comparison school CTE programs shared these goals. Third, 
POS may present opportunities for students to accrue college credits in credit-based transfer 
programs such as dual enrollment. Finally, POS must lead to a credential, certificate, or degree. 
Again, CTE programs at the comparison schools did not meet these requirements. 

Standard CTE programs might offer several of these elements but in the case of the participating 
districts in this study, only the POS offer the full set of elements. Two of these districts chose to 
create high schools dedicated to POS in the form of career academies or magnets; the third 
district integrated POS into all of its comprehensive high schools. There are other themed high 
schools in the districts besides the participating schools, including perfonning arts or 
International Baccalaureate (IB) high schools, which were usually but not exclusively entered 
through the same lottery system that selected POS students. 

In addition to being a departure from traditional CTE, POS are also different from the typical 
high school experience for college preparatory students. POS provide the opportunity for college 
preparatory students accustomed to standard academic courses to be challenged in hands-on, 
career-based ways. POS can also decrease the financial burden of higher education by allowing 
students to earn postsecondary credits in high school. 

Theory of Change 

POS change several aspects of the default high school experience. POS upgrade CTE, a program 
that has been perennially popular among students but traditionally considered less academically 
rigorous than college preparatory tracks in high school, and directed towards students not 
interested in attending college. As laid out in Perkins IV, POS infuse CTE with standards-based 
academics and seamless postsecondary connections; if implemented as mandated, POS may 
potentially lead to improved academic achievement, high school completion, and postsecondary 
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matriculation. The implicit theory of change is that infusing CTE programs with rigorous 
academics and postsecondary connections will (1) help students see the relevance of the 
academic subjects they are studying, and (2) attract a broad range of students of all levels of 
ability and interest, both of which should improve overall student outcomes. 

POS in Context: College and Career Readiness and the Underpreparedness Problem 

College and career readiness has become the federal focus of public education reform as the 
conversation around the reauthorization of the Elementary and Secondary Education Act (ESEA) 
began in 2010. In a statement made to the National Governors Association on February 27, 2012, 
President Obama stated, “The jobs of the future are increasingly going to be those with more 
than a high school degree. And I have to make a point here. When I speak about higher 
education, we’re not just talking about a four-year degree. We’re talking about somebody going 
to a community college and getting trained for that manufacturing job that now is requiring 
somebody walking through the door, handling a million-dollar piece of equipment.” Federal 
education policy reflects a clear goal that every student should graduate from high school ready 
for college and a career, regardless of their income, race, ethnic or language background, or 
disability status. 

Most states have been actively working to improve college and career readiness both 
academically (e.g., meeting high standards in traditional academic subjects; see Achieve, Inc., 
2010; Allen & Sconing, 2005; Rolfhus, Decker, Brite, & Gregory, 2010; Southern Regional 
Education Board, 2010) and technically (e.g., rigorous assessments of technical skills; 
competency in 21st-century or workplace readiness skills; see Institute for a Competitive 
Workforce, 2008; National Research Council, 2012; Partnership for 21st-Century Skills, 2009; 
Saunders & Chrisman, 2011; Stone & Lewis, 2012). However, most students remain poorly 
prepared for either. Striking gaps exist between the knowledge and skills students gain in school 
and the knowledge and skills expected by colleges and employers. 

ACT (2013) recently found that only 26% of high school students taking the ACT exam met 
college-ready benchmarks in English, reading, mathematics, and science. The Alliance for 
Excellent Education (AEE, 2008) has estimated that, nationally, about a third of all students 
entering ninth grade will graduate unprepared for college or an entry-level job, and another third 
will drop out; those most likely to do poorly are minority and low-income students. Research has 
shown that dropouts are more likely to be unemployed, collect government assistance, or spend 
time in prison (AEE, 2009; Natriello, McDill, & Pallas, 1990; Rumberger, 1987, 1995). 

Dropouts also earn far less than those with a diploma or some higher education (AEE, 2009). 
National Center for Education Statistics (NCES) data show that those with a baccalaureate 
degree earn 28% more than those with an associate degree, 53% more than those with a diploma, 
and a staggering 96% more than high school dropouts (Aud & Hannes, 2010). At the same time, 
studies by the Center on Education and the Workforce at Georgetown University show that 31% 
of individuals who acquire postsecondary technical certifications earn more than those with 
baccalaureate degrees. These data show the importance of continuing education beyond high 
school (Camevale, Smith, & Strohl, 2010). 
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For those students who do graduate and successfully enroll in some fonn of postsecondary 
education, most will be identified as in need of remediation. As few as one in five high school 
graduates are prepared for college-level classes (ACT, 2008). The National Center for Public 
Policy and Higher Education and the Southern Regional Education Board (NCPPHE & SREB, 
2010) have estimated that, despite graduating with allegedly “college-ready” high school 
transcripts, as many as 60% of all U.S. first-year college students require non-credit bearing 
remedial courses in English or math (see also Bailey, 2009). Current and former community 
college students have reported that remedial courses were not offered in ways that helped them 
succeed; they also felt unprepared to establish career goals or take the steps necessary to achieve 
them (Public Agenda, 2012). These students believed that being in programs with well-defined 
pathways and more access to career information would give them a greater chance of achieving 
their goals. 

Many underprepared students fail to complete their programs. According to Symonds, Schwartz, 
and Ferguson (2011), the on-time completion rate at four- year colleges and universities is 56%, 
and at two-year or community colleges, 29%. Most students who fail to complete college on time 
will never finish (NCPPHE & SREB, 2010; Symonds et ah, 2011). Whereas more selective four- 
year universities screen out underprepared students and have less of a readiness gap, in less 
selective four- and two-year colleges — the most affordable and likely choice of low-income, 
first-generation college students (Deil-Amen & Tevis, 2010; Bill & Melinda Gates Foundation, 
2005; Perna et ah, 2008; Venezia & Kirst, 2005) — the readiness gap is enonnous. As many as 
75% of students at two-year colleges require remediation (NCPPHE & SREB, 2010), and many 
drop out in frustration and in debt. Even colleges with liberal or open admissions may be out of 
reach or require considerable remediation in core academic subjects (Deil-Amen & Tevis, 2010). 

The academic readiness gap is echoed in the workplace, as employers continue to report that 
recent high school graduates are unprepared to pursue entry-level jobs or career training 
programs (Business Roundtable, 2009; Manufacturing Institute & Deloitte Consulting, 2011). 

The nation’s largest employers have warned of a growing skills gap and indicated a “reluctance 
to hire young people with just a high school degree. . . . Once the recession ends, they could face 
shortages of qualified workers” (Symonds et ah, 201 1, p. 4). Yet the U.S. economy remains one 
in which most jobs require at least some postsecondary education and training (AEE, 2008; 
Carnevale et ah, 2011; Carnevale et al., 2010; Ruppert, 2003). 

The high school curriculum is known to play a significant role in college access and success 
(Adelman, 1999, 2006). In response, states and districts have sought to implement college and 
career-preparatory programs like POS that address readiness gaps and prepare students for 
further education and training. Career-focused education that blends rigorous, standards-based, 
integrated academics with hands-on, challenging, career-ready skills, enriched by educational 
partnerships with industry, may help mitigate the effects of poverty and low parental educational 
attainment and positively influence students’ postsecondary aspirations. The opportunities POS 
extend to earn college credit while in high school may also shorten students’ time to degree and 
decrease the financial burden of college attendance. 
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The Federal Mandate: Career-Focused Programs of Study 


Successive reauthorizations of the Carl D. Perkins legislation that funds CTE have addressed 
how career-focused education should prepare students for both postsecondary education and the 
workplace. Perkins II (1990) required secondary and postsecondary institutions to fonn regional 
consortia and align their curricula through articulation agreements spanning two years of high 
school and two-year community college programs. These agreements led to programs known as 
2+2 programs or tech prep. Perkins II also specified that academic curricula and career-focused 
education should be integrated in order to foster the learning of advanced academic skills in 
career-oriented, technically challenging settings. Perkins IV (2006) advanced this line of policy, 
mandating the connection between career-focused education with academic core subjects and 
postsecondary preparation through POS. In the Perkins IV legislation, POS have four 
components. They: 

• must incorporate secondary education and postsecondary education elements; 

• must include coherent and rigorous content aligned with challenging academic standards 
and relevant career and technical content in a coordinated, non-duplicative progression of 
courses that align secondary education with postsecondary education to adequately 
prepare students to succeed in postsecondary education; 

• may include the opportunity for secondary education students to participate in dual or 
concurrent enrollment programs or other ways to acquire postsecondary education 
credits; and 

• must lead to an industry-recognized credential or certificate at the postsecondary level, or 
an associate or baccalaureate degree. (Perkins IV, Section 122[c][l][A]) 

POS change several aspects of the default, business-as-usual high school experience. POS infuse 
academic rigor, seamless postsecondary connections, and college preparation into career-focused 
education, which has formerly been perceived as directed toward students not interested in 
college. For example, many high school health sciences programs have offered preparation for 
CNA (Certified Nurse Assistant) certification; many students chose these programs in order to 
find employment upon or even before graduation. Now, high schools offer medical sciences POS 
that incorporate the CNA certification but also include applied medical courses that are 
integrated with college-preparatory mathematics and science; internship and training 
opportunities at local facilities (e.g., hospitals, clinics, nursing homes); community college 
courses; and additional certifications (e.g., phlebotomist, phannacy technician). Infonnation 
about the myriad careers available in medicine is an integral part of these POS. Such programs 
also encourage community service activities that hone students' skills (e.g., hosting community 
health fairs, blood pressure screenings). Graduates of such programs may then leverage these 
skills and credentials in high-demand full- or part-time jobs that can help pay for higher 
education in preparation for a broad range of potential careers (e.g., nursing, occupational or 
sports therapy, general practice, surgery, medical research). 

Prior to Perkins IV, many states and districts were implementing programs resembling POS, 
mostly in the fonn of tech prep articulation agreements between secondary and postsecondary 
institutions, then primarily community and technical colleges. Perkins IV expanded these 
agreements to include four-year colleges and universities. Driven by cross-institutional faculty 
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collaboration, articulation agreements align career-focused programs spanning secondary and 
postsecondary education in a non-duplicative progression of courses. Two mechanisms for 
earning postsecondary credit are usually employed. In the first, participating postsecondary 
institutions accept approved high school courses as the equivalent of certain introductory-level 
postsecondary courses. In the second, postsecondary institutions extend dual or concurrent 
enrollment opportunities to students, allowing them to take college courses (at a community 
college or the high school), often for both secondary and postsecondary credit. 

Review of Research on the Precursors of Programs of Study 

The business-as-usual high school experience has sometimes been characterized by ability 
tracking, with its disproportionate effect on low-income and minority students (Oakes, 2005), 
and unfocused course-taking (Powell, Farrar, & Cohen, 1985). Both lead to disengagement and 
dropout (Bridgeland, Dilulio, & Morison, 2006; Princiotta & Reyna, 2009; Rumberger & Lim, 
2008). In contrast, POS focus education on the adult world, allowing students to explore careers, 
leam career development skills, earn college credits and credentials, and engage in meaningful 
work through internships with business partners. POS also help students see the importance of 
college as a means of achieving career goals. 

When POS were mandated in 2006, the National Research Center for Career and Technical 
Education (NRCCTE) conducted a review of research on their precursors (Lewis & Kosine, 
2008). They found that programs resembling and predating POS — like (a) tech prep, (b) dual 
enrollment, and (c) career academies, described below — often share elements of POS like 
student cohorting in career-themed course sequences, college -preparatory academics, 
opportunities to earn college credits and industry-recognized credentials or certificates, and/or 
work-based learning opportunities such as internships. 

Tech Prep 

One of the goals of tech prep was to eliminate redundancies across and between high school and 
community college courses and provide high school students with clear paths to postsecondary 
education and related careers. By relocating foundational coursework at the high school level, 
tech prep allowed community colleges to teach the more advanced courses thought to be 
necessary for highly technical occupations. Research on the effectiveness of tech prep has shown 
mixed results. 

Bragg et al. (2002) conducted a longitudinal study of eight well-established tech prep consortia. 
They found that tech prep students had a significantly higher two-year college enrollment rate 
than non-tech prep students in just two of the eight consortia. Across all consortia, students from 
both groups attended community college in high numbers but most needed to take remedial 
courses in academic subjects; few from either group had earned a certificate or degree three to 
four years later. 

An early evaluation of tech prep implementation across 10 consortia surveyed state and local 
administrators, estimating that only 8% of high school students participated in tech prep 
(Hershey, Silverberg, Owens, & Hulsey, 1998). This study found that parents and students often 
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balked at strictly defined sequences of courses explicitly intended to prepare students for 
community college; these course sequences were perceived as eliminating the possibility of 
attending a four-year college or university. Urquiola et al. (1997) found that, once out of high 
school, tech prep students did not always take advantage of the program’s articulated coursework 
by proceeding to more advanced courses at the community college level; instead, students often 
had to retake the articulated courses in college, either because the credit was not automatically 
recorded on their college transcript, or because students were convinced to retake courses with 
specific college faculty. 

Hershey et al. (1998) described the diverse ways in which tech prep was being implemented 
around the country. Perkins II allowed broad discretion in meeting the requirements of tech prep 
in order to allow various models to emerge. However, Hershey et al. found that many tech prep 
elements (e.g., course articulation between secondary and postsecondary institutions) were being 
implemented piecemeal and without regard to those other elements that would go into 
constituting a coherent program for students to consciously choose. Hershey et al. recommended: 

Our five-year evaluation suggests that prospects for tech prep to change educational pathways 
and success of students... will be enhanced if federal and state education agencies renew their 
emphasis on developing structured, focused programs of study with a strong career theme, 
meaningful integration between technical and academic curricula, and a close link between high 
school and postsecondary stages of the program, (p. 130) 

Since its inception, participation in tech prep has not been systematically flagged in most 
national-level education databases. Research on tech prep (with the notable exception of the 
2002 Bragg et al. study) has largely involved detailed student-level transcript work or relying on 
student self-reported data. Stone and Aliaga (2003) examined student high school course-taking 
from the National Longitudinal Survey of Youth (NLSY:97), finding no relationship between 
tech prep participation and high school GPA. 

Dual Enrollment Programs 

Dual enrollment refers to credit -based transition programs in which high school students take 
college-level classes, either in high school or at a local college, and taught by either accredited 
high school instructors or college faculty. In some dual enrollment programs, students receive 
only college credit; in others, they receive both high school and college credit. High schools and 
colleges collaborate to provide dual enrollment opportunities for a number of compelling 
reasons. First, such programs may instill rigor and purpose in the senior year of high school, a 
focus of some concern (National Commission on the High School Senior Year, 2001). Second, 
they may serve to not just demystify the college experience, but also shorten the time needed to 
finish a degree or certificate, thereby encouraging persistence and completion. Third, dual 
enrollment may also be used for enrichment or credit retrieval. Fourth, for many budget-strapped 
districts, dual enrollment programs give students access to courses that are unavailable to them in 
high school. Finally, colleges see dual enrollment as a valuable recruitment tool. 

According to NCES data, 82% of U.S. high schools reported having students enrolled in dual 
enrollment courses (Thomas, Marken, Gray, & Lewis, 2013). Of these high schools, 93% had 
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students enrolled in academic dual enrollment courses, and 59% of the schools had students 
enrolled in CTE dual enrollment courses (Thomas et ah, 2013). A total of 1,277,100 high school 
students took a college-credit course within a dual enrollment program during 2010-11 (Marken, 
Gray, & Lewis, 2013) for a total of 2,036,700 dually enrolled courses taken. Of these 2 million 
courses, 70.5% were academic courses, and 29.5% were CTE courses. On the other side of the 
equation, 53% of U.S. postsecondary institutions pennitted high school students to take college 
courses in 2010-1 1 (Marken et al., 2013). 

Research on CTE student participation in dual enrollment conducted in Florida and New York 
City (Karp, Calcagno, Hughes, Jeong, & Bailey, 2007) showed that participating students in 
Florida were more likely to graduate from high school, enroll in a 4-year college or university, 
enroll full time, and persist to the second year compared to similar students in the same grade 
cohorts. Florida CTE students who participated in dual enrollment also enjoyed higher GPAs in 
high school and earned a higher number of credits in college. Dual enrollment CTE students in 
New York City were found to be more likely than their counterparts to pursue a baccalaureate 
degree, have a higher GPA in the first semester of college, and have earned more credits three 
and a half years after enrolling in college (Karp et al., 2007). Using secondary data, An (2013) 
found that participation in dual enrollment programs improved college degree attainment, 
particularly for first-generation college students. 

A three-year study tracking outcomes of thousands of students in career-focused dual enrollment 
programs in California found that participants were more likely to graduate from high school, 
transition to a four-year (versus a two-year) college, persist in postsecondary education, and 
accumulate more college credits, and were less likely to take remedial or developmental 
education courses than comparison students (Hughes, Rodriguez, Edwards, & Belfield, 2012). 

Career Academies 

According to the National Career Academy Coalition (NCAC), career academies are 
characterized by (a) a small learning community, comprised of a group of students within the 
larger high school who take classes together for at least two years and are taught by a team of 
teachers from different disciplines; (b) a college-preparatory curriculum with a career theme that 
helps students see the relationships among academic subjects and the application of academics to 
a broad field of work; and (c) partnerships with employers, the community, and local colleges 
that generate resources to improve student motivation and achievement. 1 

Stem, Wu, Dayton, and Maul (2007) summarized outcomes from multiple research studies of 
career academies up to and including MDRC’s long-term randomized controlled trial (RCT) 
study of career academies (cf. Kemple & Rock, 1996), the results of which we describe 
separately and in greater detail below. Stern et al. found that across six large-scale studies using 
longitudinal data to compare student achievement between academy and non-academy students 
in the same high schools, academy students outperformed their peers on various measures of 
high school achievement. For example, career academy students had higher attendance rates 
(Elliot, Hanser, & Gilroy, 2000; Hayward & Talmadge, 1985; McPartland, Balfanz, Jordan, & 


1 http://www.ncacinc.com/index.phr>?option=com content&view=article&id=10&Itemid=21 
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Legters, 1998; Stem, Dayton, Paik, & Weisberg, 1989), lower attrition rates (Elliot et al., 2000; 
Reller, 1984; Stem et al., 1989), better grades (Elliot et al., 2000; Hayward & Talmadge, 1985; 
Maxwell & Rubin, 2000; Stem et al., 1989), and a higher likelihood of completing high school 
(Hayward & Talmadge, 1985; Maxwell & Rubin, 2000) than their non-academy counterparts. 

Some of these studies extended beyond high school into postsecondary education, where results 
were less consistent but still tended to favor academy students. Reller (1987) found that higher 
numbers of academy students entered postsecondary education than non-academy students, but a 
follow-up survey two years after graduation found no significant differences in employment 
status or wages earned. Stem, Raby, and Dayton (1992) found no consistent differences between 
academy and non-academy students on measures of postsecondary attendance or wages earned, 
although academy students tended to be employed more hours per week. Maxwell and Rubin 
(2000) found more academy students entering postsecondary education than non-academy 
students: a subsequent study at one university indicated that academy students needed remedial 
education less often and were more likely to complete baccalaureate degrees (Maxwell, 2001). 

Stem et al. (2007) examined academy and non-academy student perfonnance across 26 student 
cohorts from seven career academies in five high schools in California, with mixed results. For 
the most part, they found no significant differences between the groups on measures of GPA, 
credits earned, and suspensions. In some individual cases, academy students achieved better 
results, and in others, the non-academy students did. Given that by law, at least 50% of these 
California Partnership Academy students had to meet state definitions of at risk, including low 
socioeconomic status or low test scores (Dayton, Hester, & Stem, 2011), the parity in results 
affirmed the use of career academies. Stern et al. found consistent positive outcomes for 
academy students in attendance. Descriptive data had already demonstrated that fewer academy 
students left the district for unknown reasons (often a proxy for dropout). Both of these results 
led the authors to hypothesize that if some students leave school because they are struggling, and 
academies are better at keeping students in school, then those struggling students who are in 
academies will stay, bringing down the apparent effectiveness of the academies but in fact 
improving educational outcomes for at-risk students. 

As a result of the growth of career academies nationally, career academy networks have been 
developed, perhaps most prominently the National Academy Foundation (NAF), which supports 
schools and districts implementing academies structured around any of NAF’s five career themes 
(i.e., Finance, Hospitality & Tourism, Information Technology, Health Sciences, Engineering). 
Orr, Bailey, Hughes, Kienzl, and Karp (2007) sought to detennine whether the NAF experience 
was positively associated with student achievement, selecting for study NAF academies that had 
been in operation for 10 years as a measure of sustained implementation. Orr et al. found that 
NAF students were more likely to have participated in a paid internship experience in high 
school and found it worthwhile. They found no significant differences between NAF and non- 
NAF groups on high school attendance, GPA, or students’ belief that they would complete a 
four-year degree. 

Orr et al. (2007) also found an interesting difference between students in the two themed 
academies that participated in the study: Students in the Academy of Finance were significantly 
more likely to be planning to attend a four- year college, whereas students in the Academy of 
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Travel and Tourism were significantly more likely to be planning to work after high school and 
not attend college. These outcomes may be due to the different career opportunities available to 
students in these respective industries. For those without a college degree, the finance industry 
offers limited promotion opportunities and lower potential earnings. However, the tourism 
industry offers many opportunities to work, and the authors pointed out that it is common 
practice in tourism for employers to pay for employees’ college tuition. This study highlighted 
the important influence of academy theme on post-high school educational and career plans. 

The studies reviewed here show mixed but promising results for students in career academies. 
However, none of these studies were able to control for all unobserved differences between the 
groups that might influence the results (e.g., motivation). Many academies require students to 
apply to participate; career academy students thus may exhibit greater motivation — to join the 
academies in the first place, or to succeed in their studies — than non-academy students. 

The issue of selection bias was best addressed in MDRC’s longitudinal RCT study of career 
academies, which was undertaken to more definitively estimate the impact of career academies 
on student achievement. Kemple and his colleagues were able to randomly assign eligible 
applicants to career academy and control conditions at nine sites, thus ensuring that unobserved 
differences between students were randomly distributed across the two conditions. This study 
presents the best available evidence of the impact of career academies, having met the What 
Works Clearinghouse (U.S. Department of Education, 2013a) rigorous evidence standards. 

Kemple and Snipes (2000) demonstrated that academies are a promising method of organizing 
the high school experience and curriculum for all students, but with particular benefits for at-risk 
students. They showed that among students identified as at the highest risk of dropping out, 
career academies significantly reduced dropout rates, increased attendance, increased the credits 
students could earn toward graduation, and increased the likelihood of their applying to college. 
Among those groups identified as at lower risk of dropping out, the effect of career academies 
was limited to increasing the likelihood of a student having a dual academic and career-oriented 
curricular concentration in high school. Such students were equally likely as their low-risk non- 
academy counterparts to complete the core academic curriculum, but they outperformed their 
counterparts on completing career-oriented course sequences. 

Eight years after study participants graduated from high school, Kemple found no difference 
between academy and non-academy students with respect to completion of a postsecondary 
credential. However, career academy students earned an average of 1 1% percent more annually 
than their non-academy counterparts (Kemple & Willner, 2008). 

National Assessment of CTE 

Reauthorizations of Perkins legislation have included an independent assessment of the law’s 
implementation and outcomes; however, there is an inevitable lag between a policy change and 
the availability of data under the new policy. An interim report of the independent assessment 
under Perkins IV (U.S. Department of Education, 2013b) described CTE participation as of 
2004 — two years before the passage of Perkins IV. As noted by the authors, the ability to 
evaluate Perkins IV is limited by this lag, and therefore the impact estimates in the report 
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“mainly reflect conditions of CTE that were in place at the start of the 2006 act, not those that 
unfolded subsequently” (p. viii). In spite of this shortcoming, the interim report noted (1) that 
students who participate in CTE in high school attend college at the same rates as nonCTE 
participants, and (2) among students who do not attend college, those who took CTE in high 
school are more likely to find skilled jobs than those students who did not take any CTE in high 
school. 

Two studies of CTE were included for the assessment, both of which also necessarily included 
pre-Perkins IV, pre-POS data. Results were mixed: An analysis of Education Longitudinal Study 
(ELS:2002) data found no relationship between the CTE course-taking of the Class of 2004 and 
their achievement based on a standardized math assessment (Bozick & Dalton, 2013). Similarly, 
a study of CTE high school attendance in a large urban school district (Neild, Boccanfuso, & 
Byrnes, 2013) found no relationship between CTE high school attendance and achievement. 
However, the Neild study did find a positive relationship between CTE high school attendance 
and math course-taking and graduation. The interim assessment issued two recommendations of 
note: first, CTE accountability systems must be improved because current perfonnance data do 
not include the many students who take CTE classes but do not complete program sequences. 
Second, research on the effects of CTE needs to disaggregate analyses by program and 
implementation quality in order to provide useful information to the field. 

Prior Research on POS 

The NRCCTE embarked on three field-based longitudinal studies of POS as soon as POS were 
mandated under Perkins IV. These studies probed the impact of POS on various secondary and 
postsecondary outcomes. They included (a) an exploration of South Carolina’s Education and 
Economic Development Act (EEDA), (b) a three-site study of well-established POS at the 
postsecondary level, and (c) the current study. Below, we discuss findings from the first two 
studies. 

Hammond et al. (2013) conducted a quasi-experimental, mixed-methods multi-site study of 
South Carolina’s EEDA, a career-focused school reform policy that strongly resembled POS. 
They found an increase over time in the amount of career planning activities and career-based 
counseling students received. Although they found a large increase in dual credit course-taking 
for CTE students, but not for non-CTE students, both groups earned similar numbers of credits. 
Analyses of state longitudinal data showed that CTE course sequence takers were more likely to 
plan to enroll at a two-year college after graduation, whereas those not in such sequences were 
more likely to plan to enroll at a four- year college. Students at low-poverty schools were more 
likely to report planning to complete a baccalaureate degree or higher, compared to students in 
moderate- or high-poverty schools. However, between the study’s two cohorts, the Class of 2009 
and the Class of 201 1, an increased percentage of students from both high- and low-poverty 
schools stated that they planned to complete a baccalaureate or master’s degree. 

Alfeld and Bhattacharya (2013) examined three well-established postsecondary-level POS, 
which they defined as tightly linked high school-to-community college career pathway programs. 
From 2009 to 2012, including the transition out of high school into college or work, a sample of 
213 high school juniors and seniors enrolled in POS at the three sites were tracked to examine 
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their progression through their POS. Although only 30% of students continued in the same POS 
area in either college or work, POS participation appeared to benefit students. High school 
transcript analyses showed a positive relationship between POS credits, academic credits, and 
grades. Further, the majority of students reported that being in a POS made them more motivated 
to stay in school and better prepared to make choices about college and career. In longitudinal 
analyses controlling for high school GPA, the number of high school POS courses taken was 
significantly related to (a) staying in the same career cluster in college and (b) earning a college 
credential. The number of high school dual credits earned was also significantly negatively 
related to remedial courses in college and positively related to college GPA, staying in the same 
cluster, and earning a credential. Having a positive attitude toward the POS in high school was 
related to the number of credits students earned in the first year of college. Of the 65% of the 
original sample located in 2012, most were still enrolled in two- or four- year colleges. 

The current study takes a mixed-method, longitudinal approach, examining POS student 
achievement compared to well-matched comparison groups in the same districts. The districts 
participating in this study serve large numbers of low-income and minority students — 
populations often targeted for career academy experiences that boost engagement and 
achievement (Dayton et ah, 2011). 


Research Questions 

Programs of study — which offer seamless high school-to-college connections and standards- 
based academics linked to an engaging real-world career focus and business and community 
involvement — may help ensure that our nation meets the goal of leading the world in the 
proportion of college graduates it produces by 2020, including those with sub-baccalaureate 
certificates and degrees. This is the intervention we examined for five years in three different 
state and district contexts. The following are the research questions that flowed from Perkins IV 
and our review of the literature, and that guided this longitudinal study. 

1. To what extent does participation in a POS lead to improved student achievement compared to 
well-matched comparison groups? Specifically, to what extent does POS increase student: 

a. academic achievement? 

b. technical skills achievement? 

c. high school completion? 

d. completion of coursework leading to college credits? 

2. How do POS differ from the traditional high school experience of the comparison group 
students? 
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CHAPTER TWO: Method 


This study employed a multi-method, longitudinal, quasi-experimental design to investigate the 
relationship between participation in federally mandated POS and the high school outcomes 
established in the Perkins legislation for assessing the effectiveness of POS. The study consists 
of three strands employing slightly different research designs and separate analyses, one for each 
participating district. We employed this tripartite design because (a) the districts operate in very 
different contexts, with quite different state policy contexts; (b) due to contextual differences, the 
means of creating samples differed across sites; and (c) due to differences in available data, the 
measures we used to arrive at reported outcomes varied. Further, one district (South) began 
participating in the study well after the others, meaning that some data could not be collected 
there (e.g., a ninth-grade survey, classroom observations and interviews for students’ first three 
years of high school). In this section, we describe the study’s sample, measures, and procedures 
by district. 


Participants 

This study began two years after the passage of Perkins IV. At that time, not all districts or 
education agencies had operational POS functioning at a high enough level to warrant intense 
study. But because federal legislation is often written based on field-generated innovations and 
best practices, we knew that there were operational and even successful implementations of CTE 
programs resembling POS in all regions of the country. In searching for sites, we thus looked for 
districts in which POS were being well implemented in spirit if not in name. An additional key 
selection criterion was that districts had to employ a random-assignment lottery to place eligible 
eighth-grade applicants in high school POS. Our goal in establishing this criterion was to 
construct a randomized controlled trial (RCT; Shadish, Cook, & Campbell, 2002). 

After several months of search and negotiation, we began our study with two districts. The first 
was West District, which held a lottery in the spring of 2008 for eighth-grade applicants seeking 
admission to its POS high schools in the fall of 2008 (the start of our student cohort’s ninth- 
grade year). The second was East District. Although East did not hold a lottery in 2008 due to 
under-enrollment in its wall-to-wall POS high school, Blue Academy, we decided to include the 
district in the study in part because of the quality of its programs and in part due to its expressed 
desire to participate. At the end of the first year of the study (the 2008-2009 academic year), 
South District approached the NRCCTE regarding participation in its portfolio of POS research. 
Based on South’s strong POS implementation and lottery-based admissions system, we added 
this third district. However, because of the late addition of the district to the study, we were not 
able to collect qualitative site visit data until the study’s fourth year (2011-2012; our cohort’s 
twelfth-grade year); we did collect systems data for our cohort’s four full years of high school. 

Table 2.1 presents important district characteristics during our cohort’s ninth-grade year (2008- 
2009) compared to national averages in the same year. In addition, Table 2.1 also includes U.S. 
Census information on the counties served by these districts as indicators of the condition of 
their local economies. In noting differences between our three districts and national averages, 
note that the national average includes many rural districts. Our participating districts are located 
in areas classified as urban by the U.S. Census. 
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Table 2.1 


Selected characteristics of participating school districts and their populations, 2008-2009 


Characteristics, 2008-2009 

U.S. 

West 

District 

East 

District 

South 

District 

District Characteristics 





Minority students (%) 

45. 0 a 

64.7 

66.3 

60.0 

Free lunch eligible (%) 

41.8 

42.5 

48.7 

43.0 

Student/teacher ratio 

14.9 

17.2 

14.3 

12.9 

Per pupil expenditure 11 

Population Characteristics 

$10,955 

$7,546 

$8,794 

$9,296 

Total population 


1,800,000 

800,000 

1,200,000 

Population density per square 
mile 

81 

241 

1,744 

650 

Minority (%) c 

34.2 

48.2 

44.7 

36.5 

All persons living below the 
poverty line (%) 

13.6 

11.2 

11.7 

12.2 

Children under 1 8 living below 
the poverty line (%) 

18.9 

15.9 

15.6 

18.4 

Unemployment rate d 

4.9 

5.8 

5.7 

4.9 

Households receiving public 
assistance (%) 

2.4 

2.3 

1.4 

1.1 

Median household income 

$51,369 

$55,767 

$55,263 

$52,420 

Persons age 25 or over with a 
high school diploma (%) 

84.9 

83.0 

89.2 

86.9 


Sources. National averages of district characteristics were obtained from the U.S. Department of Education 
Common Core of Data (CCD) website, http://nces.ed.gov/ccd/ . District characteristics data were obtained from 
district websites or reports and the CCD. Population characteristics were based on U.S. Census data (2000 Census 
and interim updates from Current Population Reports or the American Community Survey) obtained from the U.S. 
Census website, http://www.census.gov/ . 

Note. Some of the figures have been rounded to protect district anonymity. 

a Sum of number of Native American, Black, Latino, Asian, and multiracial students divided by the total number of 
students with reported race/ethnicity data. 

b Per pupil expenditures may not be comparable because districts may vary in their definition of what to include in 
total expenditure per student. 

Calculated by subtracting the percent White (one race only) who are not Latino from 100%. 

"'Unemployment rates reflect pre-recession conditions. 

West District, located in a Western state, serves a large urban student population, over 60% of 
whom self-identify as ethnic minorities, which is much higher than the national average (see 
Table 2.1). In 2008-2009, the percentage of West’s students who were eligible for the federal 
free or reduced-price lunch program (F/RL) was similar to national eligibility rates. Prior to the 
2009 recession, unemployment rates were higher than the national average, as were household 
median incomes. East District is located in a large city east of the Mississippi River. In 2008- 
2009, the students who self-identified as ethnic minorities exceeded the national average, as did 
the percentage of students eligible for F/RL (see Table 2.1). Unemployment rates were higher 
than the national average, but so were household median incomes. South District serves a 
geographically large area located in a southern state. In 2008-2009, the percentage of students 
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who self-identified as minorities was higher than the national average, and the percentage of 
students who qualified for F/RL was similar to the national percentage (see Table 2.1). 
Unemployment rates in the South District area were the same as the national average at the time, 
and household median incomes were slightly higher. In sum, the public schools in these urban 
districts tended to serve predominantly minority students, many of whom lived in poverty. 

Sampling Procedures 

Once we selected our three sites, we employed slightly different methods to select study 
participants in each. In West District, although a lottery was used to assign students to POS, 
built-in lottery preferences (described below) made a pure RCT impossible to construct. 
However, we used the district’s lottery results to construct intervention and comparison groups. 
In East District, where no lottery was held, district personnel identified a well-matched 
comparison group to participate in the study. In South District, we discovered that lottery 
preferences accounted for over 90% of student assignments to POS and other choice programs; 
as such, student assignments were not random. We included in our South sample all students 
who were eligible for and had applied to the school choice lottery, which included POS as well 
as other choice programs such as International Baccalaureate (IB) and perfonning arts programs. 
Our total sample included 6,638 students across the three districts. We assigned students to the 
intervention or comparison group according to their ninth-grade program enrollment. We 
describe the procedures used in each district in greater detail below. 

West District’s Baseline Student Sample 

A lottery is held every spring in West District for the district’s oversubscribed high school choice 
options. These include its performing arts high school as well as its five “wall-to-wall POS” high 
schools, where all students are in a POS. At the time we chose sites, West District was in an 
expansion phase, and two of these POS high schools had only just opened (Fall 2009). We chose 
to exclude these two schools in part because they were so new, and in part because the district 
believed that they needed to get started without the burden of participation in a research study. 
Thus only three of the five POS high schools comprised the intervention condition. 

Like many districts, West employed a number of lottery preferences that served to minimize 
student travel and maximize student diversity across schools. The district reserved up to 25% of 
the available seats in any program for sibling preference, up to 25% for geographical preference, 
and up to 25% for feeder preference. The remainder of the seats available — and those not taken 
in the three preference lotteries — were utilized in a general lottery. The district could not provide 
us with infonnation on how many or which students were accepted into the POS high schools as 
a result of these preferences, except to say that in most cases, reserved seats were not all taken by 
preference students, thus returning untaken seats into the general lottery. However, due to this 
lack of infonnation, we could not estimate the existence or extent of bias in the sample and were 
therefore unable to construct an RCT. All students who were eligible for and participated in the 
lottery for the three wall-to-wall POS high schools made up the sample. 
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Identification of the Student Sample for West District 

One year after the lottery held in Spring 2008, we received anonymized student records for all 
applicants to any of the three participating POS high schools. Over 4,000 cases were included in 
the original dataset. First, we removed non-eligible applicants and duplicate records (these 
included students who had applied to more than one POS high school). This left 2,466 records. 
Another 391 cases were removed because they were assigned to the new POS high schools that 
were not participating in the study. Finally, 24 cases were removed due to coding errors and 47 
were removed because although the students had been assigned to one of the three participating 
POS schools, their programs did not qualify as POS (e.g., cosmetology, which in this district had 
no postsecondary elements or alignment). This left a baseline sample of 2,004 students, 1,175 of 
whom were in the intervention group (i.e., enrolled in one of the three POS high schools in ninth 
grade), and 829 of whom were in the comparison group. 

Background Characteristics of the West District Student Sample 

We compared student background characteristics of this sample at baseline, the end of 2008. 
Cross tabulations and chi-square analyses were performed on gender, race/ethnicity, participation 
in the F/RL program (a proxy for poverty or socioeconomic status [SES]), Limited English 
Proficiency (LEP) status, and special education status (i.e., the presence of an individualized 
education program or IEP). School engagement was measured using eighth-grade average daily 
attendance and the variable Number of disciplinary occurrences, which was collapsed into three 
categories: (1) no occurrences, (2) one to two occurrences, and (3) more than three occurrences. 
Prior academic achievement was established using results of state exams taken in eighth grade. 
Independent samples /-tests were perfonned to compare average daily attendance and scaled 
scores in reading and math. 

Table 2.2 shows that there were significantly more males in the intervention group than in the 
comparison group (49.5% versus 32.3%). At the same time, there were relatively more females 
than males in each group and in the sample as a whole, meaning that more female than male 
students applied for these programs. There were also significantly fewer students in the 
intervention group with an IEP (2.8% versus 5.9%). 
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Table 2.2 


Baseline characteristics of sample, West District 


Characteristics 

Intervention 
(A = 1,175) 
(%) 

Comparison 
(A =829) 
(%) 

Total 

(A =2,004) 
(%) 

Gender 

Male 

49.5 

32.3 

42.4 

Female 

50.5 

67.7 

57.6 

Race/Ethnicity 

Black 

10.7 

8.4 

9.8 

White 

40.9 

39.7 

40.4 

Latino 

32.4 

35.0 

33.5 

Asian 

15.1 

16.0 

15.5 

Native American 

0.8 

0.8 

0.8 

Poverty 

F/RL-eligible 

21 A 

28.1 

27.7 

Limited English Proficiency 

Yes 

2.2 

2.8 

2.4 

Has Individual Education Plan 

Yes 

— ^ ** 

2.8 

5.9 

4.1 

School Ensasement 
# Disciplinary incidents 

None 

76.0 

76.4 

76.1 

One to two 

18.0 

18.8 

18.3 

More than three 

6.1 

4.8 

I 

5.5 


Note. Comparisons are based on cross tabulations and chi-square analyses, "p < .01, *** p < .001. 


There were no statistically significant differences in eighth-grade average daily attendance (see 
Table 2.3) or eighth-grade achievement. The average eighth-grade daily attendance rate for both 
groups was 97%. 

Table 2.3 

Baseline attendance and prior achievement of sample, West District 

Intervention Comparison Total 



A 

M (SD) 

A 

M (SD) 

A 

M (SD) 

Grade 8 attendance 

1,093 

96.7% (0.03) 

811 

97.0% (0.03) 

1,904 

96.9% (0.03) 

Grade 8 pre-test math 

1,091 

368.33 (75.84) 

816 

369.20 (80.70) 

1,907 

368.70 (77.93) 

Grade 8 pre-test reading 

1,091 

344.18 (54.14) 

816 

347.32 (57.58) 

1,907 

345.52 (55.64) 


Note. Independent samples r-tests were performed to compare means. 


East District's Baseline Student Sample 

The intervention group in East District consisted of the students who began ninth grade at the 
district’s one wall-to-wall technology POS high school. Usually, students were selected to attend 
this high school through a lottery, like the POS high schools in West District. However, the 
school was expanding its enrollment in the year in which our study began. As a result of this 
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expansion, there were seats for all eligible applicants, so no lottery was held. In lieu of random 
assignment, East District personnel ran a cluster analysis on the other high schools in the district, 
searching for four high schools that were most similar to the intervention school in tenns of 
student demographic variables, mobility, and prior achievement. District personnel then used 
propensity score matching (PSM) to identify a comparison group from within those four high 
schools. Propensity scores are the estimated probability that a subject is assigned to an 
intervention based on various pieces of information (Pasta, 2000). This predicted probability is 
obtained by conducting a logistic regression that predicts membership in the intervention group 
utilizing a vector of covariate predictors, in effect mitigating the effects of inherent differences 
among students in the different study conditions (Foster, 2003; Shadish et ah, 2002). Students 
with similar distributions across the covariates will have similar estimated propensity scores. 
Thus a student in the intervention group can be matched with a comparison student possessing a 
similar propensity score, yielding reduced-bias estimates of effects during analysis (Rosenbaum 
& Rubin, 1985). 

Identification of the Student Sample for East District 

The comparison group was created by the district research office after the prospect of random 
assignment was eliminated. A matrix of intervention group variables was selected on which to 
match students from the comparison schools identified in the cluster analysis: 

• Perfonnance on standardized eighth-grade reading and mathematics tests 

• Gender 

• Race/ethnicity 

• LEP status 

• Special education classification 

• F/RL eligibility (as a proxy for poverty) 

• Eighth-grade retention 

• Number of violent incidents during the school year in which the student was involved 

• Number of nonviolent incidents during the school year in which the student was involved 

• Number of unexcused absences 

• Number of excused absences 

• Number of out-of-school suspensions 

• Number of in-school suspensions 

• Mobility during eighth grade (the number of school changes) 

Once predicted probabilities were obtained, intervention students were matched with comparison 
students possessing exact probabilities when such were available, and similar probabilities with 
minimal difference when an exact match was not available. This matching procedure was 
conducted without replacement, so that a student identified as a comparison student match for a 
particular intervention student was removed from the potential comparison pool before the next 
match was conducted. The propensity match procedure was carried out twice to provide two 
matched comparison students for every intervention student, to take into account attrition over 
the duration of the study. A comprehensive file was created containing student background 
information from the eighth-grade year (2007-2008) for all intervention and comparison 
students. 
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Background Characteristics of the East District Student Sample 

As with the West District sample, cross tabulations and chi-square analyses were performed on 
gender, race/ethnicity, F/RL participation, LEP status, IEP status, and number of disciplinary 
incidents. Independent samples /-tests were performed to compare eighth-grade average daily 
attendance and eighth-grade reading and math achievement. 

Tables 2.4 and 2.5 show the results of the PSM and the resulting comparisons of the background 
characteristics of the sample. There are no significant differences between the two East District 
groups on any of the variables in either table, which confirms that the PSM procedure found 
appropriate matches for all intervention students. In keeping with overall district demographics, 
Black students made up the majority in both groups, about two-thirds of the students in both 
groups were free lunch-eligible, and the proportion of LEP and IEP students was small. Over 
75% of all students had no disciplinary incidents while in eighth grade, and there were no 
statistically significant differences among the groups in this regard. 

Table 2.4 


Baseline characteristics of sample, East District 


Characteristics 

Intervention 
(A =376) 
(%) 

Comparison 
(A =752) 
(%) 

Total 

(A= 1,228) 
(%) 

Gender 

Male 

52.1 

49.9 

50.6 

Female 

47.9 

50.1 

49.4 

Race/Ethnicity 

Black 

71.0 

70.5 

70.7 

White 

10.6 

11.2 

11.0 

Latino 

10.4 

10.6 

10.5 

Asian 

3.7 

3.6 

3.6 

Native American 

0.3 

0.1 

0.2 

Other/Multiracial 

4.0 

4.0 

4.0 

Poverty 

F/RL-eligible 

68.4 

66.4 

67.0 

Limited English Proficiency 

Yes 

4.3 

4.8 

4.6 

Has Individual Education Plan 

Yes 

2.4 

2.3 

2.3 

School Engagement 
# Disciplinary incidents 

None 

76.3 

75.9 

76.1 

One to Two 

17.0 

17.0 

17.0 

More than Three 

6.6 

7.0 

6.8 


Note. Comparisons are based on cross tabulations and chi-square analyses. 
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Table 2.5 shows no significant differences on baseline attendance or prior achievement: Both 
groups had eighth-grade average attendance rates of 98%. Reading and math achievement across 
both groups was also similar, as measured by scaled scores on East District’s end-of-tenn exams. 

Table 2.5 


Baseline attendance and prior achievement of sample, East District 



Intervention 
M(SD ) 

(A =376) 

Comparison 
M(SD ) 

(A =752) 

Total 

M(SD ) 

(A= 1,128) 

Grade 8 attendance 

98% (0.03) 

98% (0.03) 

98% (0.03) 

Grade 8 pre-test math 

363.22 (6.96) 

362.90 (7.54) 

363.00 (7.35) 

Grade 8 pre-test reading 

359.91 (7.10) 

359.69 (7.49) 

359.76 (7.36) 

Note. Independent samples T tests 

were performed to compare 

means. 



South District’s Baseline Student Sample 

South’s POS took the form of career academies housed in comprehensive high schools across the 
district; there were no wall-to-wall POS high schools. Some POS were accessible only through 
the lottery process, whereas others were available for students attending their zoned high 
schools. Unlike in the other districts, any high school in South District could contain an 
intervention student, a comparison student, or both at baseline. 

Students applied to the lottery for their school(s) of choice during the spring of their eighth-grade 
year. As in West District, sibling, geographic, and feeder preferences were employed, in addition 
to a principal’s preference that selected up to 20% of all eligible applicants who applied to the 
school as their first choice. In our analysis of the district’s lottery assignments, we concluded that 
94.9% of all eligible applicants were assigned to a choice program, either (1) through one of the 
district’s preference mechanisms, or (2) because there were not enough applicants to a given 
program to conduct a lottery. For this reason, we abandoned the use of an RCT in this district. 
Instead, we used lottery outcomes to design a quasi-experiment consisting of a sample of 
students who applied to and were eligible to participate in the district’s choice programs. 

As West District and South District show, district-run lotteries are designed to provide desirable 
opportunities to the maximum number of students seeking them. District lotteries are not 
designed to facilitate researcher-driven RCTs. More is said about this in the Conclusion, below. 

Identification of the Student Sample for South District 

In late 2009, we received anonymized student records for all applicants to South District’s choice 
programs. Over 6,500 cases were included in the original dataset. First, we removed all 
applicants who were not in ninth grade in 2008-2009, leaving 5,123 records. Another 1,280 cases 
were removed because they were missing a school name for 2008-2009 assignment, leaving 
3,843 records. The lottery records showed that 129 applicants were not eligible for any of their 
three choices; they were removed as well. Finally, 208 cases were removed because they had no 
course-taking records to date, which at that time included the ninth and 10th grades. This left a 
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baseline sample of 3,506 students, 1,985 of whom were in the intervention group (i.e., enrolled 
in a POS in ninth grade), and 1,521 of whom were in the comparison group. 

Background Characteristics of the South District Student Sample 

We compared student background characteristics of this sample at baseline. However, eighth 
grade information on students with an IEP and students eligible for the F/RL program was not 
available; we used the tenth grade data instead. Cross tabulations and chi-square analyses were 
performed on these characteristics as in the other districts. Independent samples /-tests were 
performed to compare average daily attendance and scaled scores in reading and math. 

Table 2.6 shows that there were significantly more Native American students, students eligible 
for the free lunch program, LEP students, and students with an IEP in the intervention group than 
in the comparison group. There were also significantly fewer Asian students in the intervention 
group at South District. 

Table 2.6 


Baseline characteristics of sample, South District 


Characteristics 

Intervention 
(A = 1,985) 
(%) 

Comparison 
(A =1,521) 
(%) 

Total 

(A =3,506) 
(%) 

Gender 

Male 

48.7 

45.8 

47.5 

Female 

51.3 

54.2 

52.5 

Race/Ethnicity 

Black 

19.6 

18.3 

19.1 

White 

49.9 

50.0 

49.9 

Latino 

22.4 

20.4 

21.5 

Asian 


7.0 

5.3 

Native American 

0.8* 

0.2 

0.5 

Other/Multiracial 

3.4 

4.0 

3.7 

Poverty 

Eligible for F/RL 

32 7*** 

25.1 

28.9 

Limited English Proficiency 

Yes 

5.7* 

4.1 

5.0 

Has Individual Education Plan 

Yes 

5.5** 

3.2 

4.5 

School Ensasement 
# Disciplinary incidents 

None 

93.6 

92.5 

93.1 

One to Two 

3.0 

4.1 

3.5 

More than Three 

3.5 

3.4 

3.4 


Note. Comparisons are based on cross tabulations and chi-square analyses. 
$ && $$$ 
p<. 05 , P < . 01 , £><. 001 . 
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As shown in Table 2.7, the intervention group had significantly lower eighth-grade achievement 
as measured by state test scores. This could be attributable to the fact that applicants to programs 
such as IB and focused math and science programs were among those in the comparison group. 
These differences were controlled for in the outcomes analyses. The average eighth-grade daily 
attendance rate for both groups was 97.4%. 


Table 2.7 


Baseline attendance and prior achievement of sample, South District 




Intervention 

Comparison 


Total 


N 

M(SD) 

N 

M(SD) 

N 

M(SD) 

Grade 8 attendance 

1,808 

97.4% (0.03) 

1,287 

97.4% (0.03) 

3,095 

97.4% (0.03) 

Grade 8 pre-test math 

1,804 

328.82 (38.46)*** 

1,292 

344.69 (46.59) 

3,096 

335.44 (42.76) 

Grade 8 pre-test reading 

1,804 

348.56 (34.03)*** 

1,292 

357.45 (40.74) 

3,096 

352.27 (37.23) 


Note. Independent samples r-tests were performed to compare means. p < .00 1 . 


Posthoc Groups for Assessing the Effect of Completing a POS on High School Achievement 


For the posthoc analyses in each district, we limited the samples to those students who remained 
in the district all four years and had no missing data, in order to ensure that all students had the 
same opportunity to complete a POS. By analyzing student course-taking (see Appendix A), we 
then identified two types of students with respect to their CTE course-taking: (1) POS 
completers, or those who completed the course sequence of the POS, and (2) CTE concentrators. 
The latter is a Perkins-derived accountability category denoting a student who has completed a 
certain number of credits in an occupational area. Each district had a different definition of how 
many credits were needed to earn a CTE concentration. The number of credits required for a 
CTE concentration is negotiated between the U. S. Department of Education’s Office of 
Vocational and Adult Education (USDE/OVAE) and each state. CTE concentrators are usually 
required to earn fewer CTE credits than are POS completers, and those credits do not have to be 
taken in a specific sequence. The requirement of a specific course sequence for POS completion 
is one of the ways in which POS differ from CTE concentrations; others are included in the four 
mandatory elements of POS (see Measures). 


Sampling Limitations 

In East District, a PSM procedure provided a well-matched comparison group for our self- 
selected intervention students. Limitations in East include those unmeasured characteristics that 
might influence outcomes, such as the possibility that students who self-selected to participate in 
a POS might have more motivation to succeed than others. Neither West nor South District 
randomly assigned students to POS; however, in West, all sample students shared the important 
trait of being interested and motivated to apply to be in a POS. At South, students also applied to 
other programs such as IB and performing arts programs. Outcomes from samples like these are 
only generalizable to students who are interested in being in a POS. Each sample has its 
strengths and weaknesses; taken together, study conditions reflect the range of districts across the 
country in which POS are being offered. 
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Qualitative Samples 


To help us understand the process through which POS effect change, this study employed a 
qualitative component that included (1) stakeholder interviews, (2) classroom observations, and 
(3) student surveys. We conducted 65 student interviews over the four years of the study (20-25 
in each district). Interviewed students were not representative of the entire sample because they 
were not randomly selected; we chose from among those students whose parents returned a 
signed informed consent document (see Procedures, below), or, at South, were chosen for us by 
district personnel. We also conducted interviews with teachers, counselors, and administrators at 
both the intervention and comparison schools and at the local community colleges at West and 
East Districts. South District’s late entry into the study meant that no contacts were made with 
their local community college. We conducted a total of 95 interviews in West District, 88 
interviews in East District, and 44 interviews in South District. Appendix B provides more detail 
on these interviews. 

We observed a range of academic and career-focused courses in all three districts: 62 classroom 
observations in West, 42 in East, and 21 in South (see Appendix B for more information on our 
data collection activities). The number of observations varied across districts for two reasons: 
first, West high schools tended to follow a period schedule, whereas East high schools tended to 
follow a block schedule. Although the length of the school day was the same in both districts, we 
found it difficult to schedule two interviews or observations during one course block. In essence, 
in East District, we had fewer time slots in which to schedule research activities. Second, the 
delayed entry of South District into the study meant fewer site visits there overall, resulting in 
less qualitative data collection. 

Student surveys were conducted during the students’ ninth-grade and twelfth-grade years. In 
general, student response rates were low. In West District, 43.3% of the sample responded to the 
ninth-grade survey and 37.5% responded to the twelfth-grade exit survey. In East District, 24.9% 
of sample students took the ninth-grade survey, and 36.0% took the twelfth-grade exit survey. 
Surveys were not conducted in South District. The low response rate was in part due to missteps 
at the district level: In one case, survey invitations were mailed to an older, erroneous sample 
list; many completed surveys from this mailing had to be discarded because the students were 
not in our sample. Mishaps like this occur when, in order to protect student privacy, researchers 
cede control of survey administration to the district. They are virtually an unavoidable hazard in 
conducting longitudinal research in educational settings. 

Measures 

Our outcome measures are found in district systems data and are of practical interest to schools. 
The intent of Perkins IV is to improve the achievement of CTE students in relation to state 
academic standards and to ease their transition to postsecondary education. But as with many 
federal education policies, POS requirements were written broadly in order to promote 
implementation in different state contexts and to encourage innovation. When our study began, 
the four elements written into the legislation were the sole guidelines for education agencies to 
follow in implementing POS (cf. Table 2.8). 
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Table 2.8 


The four mandated elements ofPOS 


i 

POS must incorporate secondary education and postsecondary education elements. 

ii 

POS must include coherent and rigorous content aligned with challenging academic 
standards and relevant career and technical content in a coordinated, nonduplicative 
progression of courses that align secondary education with postsecondary education to 
adequately prepare students to succeed in postsecondary education. 

iii 

POS may include the opportunity for secondary education students to participate in dual or 
concurrent enrollment programs or other ways to acquire postsecondary education credits. 

iv 

POS must lead to an industry-recognized credential or certificate at the postsecondary level, 
or an associate or baccalaureate degree. 


Source : Perkins IV, Section 122[c] [1][A] . 


We operationalized these POS elements into outcomes that capture the skills and content that 
POS were designed to address, according to Perkins IV : 

a. Academic achievement: Cumulative grade point average (GPA), STEM credits 
earned (i.e., credits in science and in higher math, defined as math above Algebra 
II), and AP credits earned 

b. Technical skills achievement: CTE GPA 

c. High school completion: Graduation 

d. Transition: Post-high school plans and completion of coursework associated with 
college credits (i.e., dual enrollment) 

The first three measures include academic and technical achievement outcomes and high school 
completion. The data for these measures were collected from the districts. An overall GPA was 
calculated from students’ course grades. On the surface, course grades seem an unreliable 
measure of achievement because of the variation in grading standards across teachers and 
schools. However, there is a strong correlation between GPA and college admission test scores 
as a predictor of college success (Stumpf & Stanley, 2002), and some studies have shown the 
superiority of GPA over such admissions tests in predicting college graduation (Atkinson & 
Geiser, 2009). Beyond educational predictive power, some studies also show that GPA predicts 
salary level and job performance; however, such studies refer to postsecondary GPA (e.g., Roth 
& Clarke, 1998). 

Higher mathematics coursework was selected as part of the STEM (science, technology, 
engineering, and mathematics) measure for several reasons. Participation and achievement in 
math are gateway indicators for positive post-high school outcomes such as postsecondary 
educational attainment and labor market success (National Research Council, 1989; U.S. 
Department of Education, 1997). In addition, the mathematics course sequence in high school is 
usually well-defined, making comparisons relatively straightforward. Math is often embedded in 
CTE coursework, making student achievement in math particularly relevant for POS. Prior 
research has shown that completing the math sequence is correlated with choosing STEM majors 
in college (Chen, 2009) as well as with college graduation and employment incomes (National 
Mathematics Advisory Panel, 2008). 
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We chose CTE GPA as the technical assessment measure because it is an acceptable measure for 
Perkins reporting, absent other measures. East had end-of-term assessments in its CTE courses, 
but comparisons were difficult because Blue Academy and comparison students tended to take 
different CTE courses and so different exams. South had industry-recognized credentials as their 
measure of technical assessment for Perkins, but the data we received were incomplete with no 
way to ascertain the nature of the missing data. Many credential-granting organizations, be they 
private industries or the state, send credentialing exam results directly to students’ homes, not to 
their high schools, adding to the difficulty of collecting the data. We chose to be consistent 
across districts by using CTE GPA. 

Transition measures capture whether students have made a plan for their next step after high 
school and begun to accrue college credits as part of that plan. Data for the first transition 
measure came from a senior exit survey conducted in May 2012, the sample’s senior year. Data 
for the latter transition measure came from systems data collected annually. 

Our qualitative research question, posed in its own right and as an explanatory complement to 
our quantitative outcomes, asks How do POS differ from the traditional high school experience 
of the comparison group students? We delimited this question in order to explore the ways in 
which the POS experience and the traditional CTE experience differed. This was the most logical 
difference to explore given the data we collected. We have fewer conclusions to draw about the 
high school experience of comparison students who took no CTE classes. 

Qualitative Measures 

In addition to exploring whether participation in a POS leads to improved student outcomes as 
compared to defined comparison groups, we believe it is useful to probe the ways in which 
improved outcomes might occur. Our interviews, observations, and student surveys were all 
designed to provide detailed infonnation on how POS were developed and implemented. We 
focused on what participants perceived to be the most important differences between the 
intervention and the comparison schools, as well as the usefulness of their school’s programs as 
preparation for postsecondary education and work. We also used these data sources to further 
explore the counterfactual experiences of students in the comparison schools in order to ensure 
that POS were not a part of the comparison group experience. 

Qualitative research involves the analytical interpretation of data collected in interviews and 
observations in order to identify concepts and relationships among concepts, then organizing 
these into an explanatory scheme (Strauss & Corbin, 1990, 1998). This is done by identifying a 
set of conceptual codes and marking the instances in which they appear in the data. More detail 
about the procedures is provided in Qualitative Analyses below. 

We also developed a plan to analyze implementation fidelity. In the best cases, information on 
the fidelity of implementation of an intervention can be integrated with outcomes analyses (U.S. 
Department of Education, 2013c). The elements of Perkins IV are the only elements from which 
we could outline acceptable ranges of variation for an analysis of implementation fidelity. 
However, those elements do not lend themselves to “levels of use” (O’Donnell, 2008) or other 
quantifiable fidelity measures; these four elements of the law function more as a yes/no checklist 
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(cf. Table 2.8). It became clear that the open, somewhat vague nature of the law’s requirements 
made it next to impossible to operationalize and measure implementation criteria. 

As education agencies began to implement POS, USDE/OVAE, in partnership with the National 
Association of State Directors of Career Technical Education Consortium (NASDCTEc), 
formulated a policy guidance framework (USDE/OVAE, 2010) to help states and other agencies 
meet POS requirements. This framework was not available to us at the time of site selection or 
early fieldwork. The framework’s ten voluntary components “support the development and 
implementation of effective programs of study” (p. 1). They include: 

1 . Legislation and Policies: Federal, state, and local legislation or administrative policies 
promote POS development and implementation. 

2. Partnerships: Ongoing relationships among education, business, and other community 
stakeholders are central to POS design, implementation, and maintenance. 

3. Professional Development: Sustained, intensive, and focused opportunities for 
administrators, teachers, and faculty foster POS design, implementation, and 
maintenance. 

4. Accountability and Evaluation Systems: Systems and strategies to gather quantitative 
and qualitative data on both POS components and student outcomes are crucial for 
ongoing efforts to development and implement POS. 

5. College and Career Readiness Standards: Content standards that define what students 
are expected to know and be able to do to enter and advance in college and/or their 
careers comprise the foundation of a POS. 

6. Course Sequences: Non-duplicative sequences of secondary and postsecondary courses 
within a POS ensure that students transition to postsecondary education without 
duplicating classes or requiring remedial coursework. 

7. Credit Transfer Agreements: Credit transfer agreements provide opportunities for 
secondary students to be awarded transcripted postsecondary credit, supported with 
formal agreements among secondary and postsecondary education systems. 

8. Guidance Counseling and Academic Advisement: Guidance counseling and academic 
advisement help students to make informed decisions about which POS to pursue. 

9. Teaching and Learning Strategies: Innovative and creative instructional approaches 
enable teachers to integrate academic and technical instruction and students to apply 
academic and technical learning in their POS coursework. 

10. Technical Skills Assessments: National, state, and/or local assessments provide ongoing 
information on the extent to which students are attaining the necessary knowledge and 
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skills for entry into and advancement in postsecondary education and careers in their 
chosen POS. 2 

Many of these components are not easily measured: For example, under the framework, 
legislation and policies supporting POS or professional development for POS teachers were 
encouraged but not required; nor were details regarding such policies provided. The components 
are also multifaceted. For example, Teaching and Learning Strategies describes several 
strategies, one of which is the implementation of career and technical student organizations 
(CTSOs), which provide engagement and leadership opportunities for students. Because CTSOs 
incorporate soft skills, they are a recommended teaching and learning strategy for POS students. 

Although USDE/OVAE elaborated on and provided policy support related to these suggested 
components, they were ultimately not enforceable because they were not part of the law. Given 
this, as well as the difficulties of quantifying them, we concluded that our analysis of 
implementation fidelity would have to be qualitative. As such, we ensured that we had codes for 
each suggested component and its subcomponents. 

This report responds to the funding agency, USDE/OVAE, by providing a picture of how the 
intervention and comparison conditions differed in relation to the policies that created and 
supported POS. By organizing the qualitative portion of this report along these lines, we have 
also sought to address the issue of implementation fidelity at the intervention high schools. 

The overarching goal of our qualitative analyses is to explore how POS differed from the 
comparison school experience, following USDE/OVAE’s ten-component framework and the 
four elements of POS outlined in the Perkins legislation. Our qualitative analyses may also 
explain and provide context for each district’s quantitative student achievement outcomes. 

Qualitative Validity and Reliability 

We established the validity of the data in four ways. First, we engaged in member checking 
(Lincoln & Guba, 1985), or getting feedback from key study participants. This provided them the 
opportunity to verify the credibility of our account, thus establishing the account as trustworthy 
and internally valid. We asked the GTE directors from each district to read our accounts of the 
fidelity of POS implementation in their district. Responses from these stakeholders confirmed 
that we had captured the contours of the narrative; in fact, no corrections were requested by these 
study participants. 

A second means of establishing validity was to triangulate the data with achievement or other 
qualitative data so that no claim arose from only one source. Third, we reached saturation in 
many of the categories we created for analysis. Saturation involves gathering data until no new 
relevant data are gathered on a category and until the categories are well developed and 
validated, reflecting the depth and breadth of the category (Josselson & Lieblich, 2003; Strauss 


2 Element #2 and component #6 are identical, as are element #3 and component #7. Therefore instead of reporting 
on 14 topics, this report only includes 12 topics but covers all of the elements and components of POS as determined 
by USDE/OVAE. 
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& Corbin, 1998). And finally, conducting mixed-method research like the present study adds 
external validity, or generalizability, to the findings. 

This study’s two co-principal investigators conducted all of the field research, including 
classroom observations and interviews. We addressed issues pertaining to the reliability of these 
data by creating detailed data collection protocols, training study team members on the protocol 
procedures, and developing a fonnal qualitative database that can, in theory, be accessed and 
analyzed by others to yield the same results. Details on reliability checks of data collection 
instruments and data analysis procedures are found in Appendix B. 

Research Design 

This was a complex, mixed-method, longitudinal study examining POS in three states. We first 
describe the quantitative approaches that we employed, followed by the qualitative design. 

Quantitative Procedures and Analyses 

We employed two different statistical approaches to the data in each district, in order to answer 
important policy questions about POS. First, we estimated the effects of enrolling in POS and 
number of CTE credits earned on GPA and graduation using an instrumental variable approach. 
In addition to that main analysis, we also addressed specific policy questions about completing a 
POS compared to other high school trajectories (i.e., a CTE concentration; see Sampling 
Procedures, Posthoc Groups ) through posthoc multiple regression analyses. Each is described in 
turn. 

Instrumental Variables Estimation 

We used an instrumental variable approach to estimate the effects of attending POS schools and 
number of CTE credits earned on GPA and graduation (see Angrist & Krueger, 200 1 ; Antonakis, 
Bendahan, Jacquart, Lalive, 2010; Greenland, 2000). Omitted variable bias is a common barrier 
to causal inference in the context of observational and quasi-experimental research. When the 
causal effect of X on Y is of interest, a parameter estimate may be biased because some or all of 
the estimated effect is attributable to omitted causes. In the context of structural or simultaneous 
equations, this problem is known as endogeneity (Antonakis et ah, 2010). The most well-known 
method for avoiding this bias is the randomized experiment, where all omitted variables are 
randomly distributed across treatment and control groups (e.g., a group that receives X and one 
that does not). In the context of large samples, this bias can also be eliminated through the use of 
an instrument for X, which is correlated with X for plausible substantive reasons, but not directly 
linked to Y. That is, X fully explains the effect between the instrument, Z, and the outcome of 
interest. In education and psychology, this instrumental variable approach is much less common 
than randomized experiments. 

The central problem stemming from omitted variable bias and non-experimental data is that 
without an instrument, X’s causal effect on Y cannot be isolated from covariance attributable to 
shared but omitted causes (see Figure 2. 1). For example, the covariance depicted in Figure 2. 1 
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Figure 2.1 

Omitted variable bias 



Note: Thick line represents an inestimable parameter. 

(denoted by a thick line) reflects omitted causes, but gets partitioned into coefficient a, creating 
bias. When an instrument is located, it is possible to partial the covariance of X with Y into 
consistent estimates of X’s effect on Y and also a residual covariance - the covariance between 
the variance in X not explained by the instrument, Z, and the variances in Y not explained by X. 
Thus, X’s effect on Y can be isolated from covariance due to omitted variables (see Figure 2.2). 


Figure 2.2 

Instrumental variable approach 



Instrumental variable estimates can be understood as well-defined effects for subsets of treated 
groups (Angrist & Krueger, 2001). Angrist (1990) used Vietnam-era lottery numbers as an 
instrument to estimate the effect of military service on earnings later in life. In this case, the 
instrument was needed because not all people were drafted into service. Many people also 
volunteered for service, meaning it was unlikely that all causes of later earnings were randomly 
distributed across those who did and did not serve. Many people could have selected into service 
for reasons that caused their later earnings. In a conventional regression, the covariation of 
service and earnings due to these shared causes would have been “forced” into the estimated 
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effect of service on earning, creating bias. In this context, using lottery numbers as an instrument 
provided an estimate of a well-defined causal effect for a subset of those who enlisted in the 
military: those who were drafted as a result of the lottery. Thus, using lottery numbers as an 
instrument for military service facilitated estimation of a Local Average Treatment Effect for 
draftees, as if they were assigned in a randomized trial (Angrist & Krueger, 2001). 

Instrumental variable approaches have also been used to estimate causal effects in the context of 
treatment non-compliance. In these cases, assignment to a treatment condition is an instrument 
for a compliance variable such as units of a protocol that were completed or completion of some 
treatment regimen (Greenland, 2000). Such models can also be conceptualized as mediator 
models, where compliance translates assignment into the outcome(s). As with the Vietnam-era 
lottery, here the instrumental variable approach helps researchers estimate a well-defined 
treatment effect for a subset of those who were assigned to a treatment condition: those who 
actually adhered to the treatment protocol. The indirect effect of assignment to a treatment on 
outcomes, through the compliance variable, is the “intent to treat” effect. 

In this study, we wanted to estimate the effects of enrollment in POS and number of CTE credits 
earned on GPA and graduation for students in three school districts. In West District, enrollment 
in POS could not be understood as stemming from a purely random process. Assignment to POS 
high schools occurred through a lottery; however, students were able to circumvent this lottery 
mechanism through preferences already described (see Sampling Procedures). Students were 
also able to select out of POS high schools, which can be seen as an analogue of non- 
compliance. Thus, it was unlikely that all omitted causes of attending a POS high school (or not) 
were randomly distributed across POS and non-POS high schools. In addition, omitted variables 
such as persistence could be expected to impact number of CTE credits earned and also the 
outcomes of interest. 

Modeling enrollment in a POS school as an instrument for number of CTE credits earned 
allowed us to address these issues, essentially combining features of the Vietnam-era draft and 
non-compliance scenarios. Use of an instrumental variable enabled us to estimate the effect of 
enrollment in a POS school for a well-defined subset of students: those whose CTE earning 
behavior changed as a result of enrollment in a POS school. For the first district, enrollment in 
POS schools could be expected to change CTE earning behavior because of the great availability 
of CTE courses at POS schools and because some students attended these schools as a result of a 
random lottery process. This approach also allowed us to estimate the effects of earning more 
CTE credits on GPA and graduation because these effects were isolated from residual 
covariances that captured their association attributable to omitted causes (see Figure 2.3). The 
substantive rationale for modeling enrollment in a POS school as an instrument for number of 
CTE credits earned was that number of CTE credits earned could be understood as a mechanism 
that translated enrollment into GPA and graduation. In other words, earning CTE credits was a 
mediator that translated enrollment in a POS school into the outcomes of interest. 

For East and South Districts, we had reason to suspect that enrollment in POS schools was not a 
function of a lottery mechanism, even for a subset of students. Therefore, it was possible that 
enrollment in a POS school did not change student CTE credit earning behavior. At East, we 
knew that students were generally able to select and enroll in the schools they chose. To address 
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Figure 2.3 

Instrumental variable estimator of the effects of enrollment in POS 



Omitted Variables 


this, the district carried out a propensity score match (previously described) to locate a 
comparison group that corresponded to the students in POS schools. In many school districts, 
this could have been expected to generate a comparison group of students that were similar to 
POS students, but earned fewer CTE credits because they were not attending a POS school. If 
this were the case, enrollment in POS schools could have functioned as an adequate instrument 
for number of CTE credits earned. However, CTE courses are extensively available at East 
District and thus the matched students may have been characterized by similar CTE credit 
earning. This could have stemmed from a combination of individual similarity to their POS 
matches and also their comparable access to CTE courses. 

In South District, the context was similar to East, except no propensity score match was 
conducted to locate a comparison group. For these two districts, we planned to test whether 
enrollment in a POS school functioned as a good instrument for number of CTE credits earned, 
and if it did not, test a conventional mediator model without the residual covariances typical of 
instrumental variable models (see the double-headed arrow in Figure 2.2). These models would 
be the same as for West, except they would not estimate the covariance between number of CTE 
credits earned and the outcomes of interest that was attributable to omitted variables. In essence, 
these models would look like common longitudinal mediator models, which are routinely tested, 
but do not support the same strength of causal inference as instrumental variable models because 
they do not correct for endogeneity (Antonakis et ah, 2010). 

Structural equations modeling. This study used structural equations modeling (SEM) to 
estimate the effects of enrollment in POS schools and number of CTE credits earned. Structural 
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equation modeling, also known as simultaneous equation modeling, is a general term that has 
been used to describe multi-equation regression models that represent causal effects between 
variables of interest (Bollen, 1989; Kline, 2010). SEM can be understood as an extension of 
general linear modeling, analysis of variance, and multiple regression. In SEM, an outcome in 
one equation may function as a predictor in another and the effects between all variables are 
estimated simultaneously. SEM often includes latent and observed variables and is often used to 
correct for measurement error and locate unbiased estimates of the effects between latent 
variables. Variables in an SEM may influence one another directly, indirectly, and reciprocally. 

We used MPlus 7.1 1, accounted for the clustering of students nested in schools, and selected 
robust weighted least squares (WLSMV) as the estimator because our graduation outcome was 
categorical (Muthen, du Toit, & Spisic, 1997). We specified a model that included enrollment in 
POS as an instrument for number of CTE credits earned. As mentioned, we theorized that 
number of CTE credits earned was the mechanism or mediator that translated enrollment in POS 
into the outcomes. To control for relevant background variables, we regressed GPA and 
graduation on covariates such as age, gender, race/ethnicity, and free lunch status, among others. 
To evaluate model fit, this study used a variety of commonly used fit indices and their rules of 
thumb (e.g., Tucker-Lewis index [TLI], comparative fit index [CFI], and root means square error 
of approximation [RMSEA]; Bollen, 1989; Browne & Cudeck, 1993; Byrne, 2001; Kline, 2010; 
Hu & Bender, 1999). 

Missing data analyses. We conducted missing data analyses because our data were longitudinal 
and attrition was a concern. We examined the missing data for each district carefully. Given 
mean differences for collected information on whether or not missing data were present, we 
rejected the assumption of missing completely at random (MCAR) and made the more relaxed 
assumption of missing at random (MAR). 

We used multiple imputation to handle the missing data and included auxiliary variables to 
satisfy the MAR assumption (Enders, 2010). Multiple imputation estimates missing values for 
variables using complete data from other variables, substitutes predicted values for missing 
information, and produces an imputed set. This is repeated multiple times and multiple imputed 
sets result. Analyses are conducted using each imputed set and the results from these analyses are 
pooled to produce overall results. 

Multiple imputation accounts for missing data by restoring the natural variability in the missing 
data and accounts for uncertainty in their estimation and imputation. This approach allows 
researchers to preserve important characteristics of the data set as a whole (e.g., variances) and 
make use of all possible information. Multiple imputation produces unbiased parameter estimates 
which reflect the uncertainty associated with estimating missing data, is robust to departures 
from normality assumptions, and provides adequate results in the presence of low sample size or 
high rates of missingness (Enders, 2010; Little & Rubin, 2002). 

Posthoc Multiple Regression Analyses 

Posthoc analyses examined the relationships between POS completer status and gains in 
academic and technical achievement as measured by: 
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• Overall GPA 

• STEM (defined as math above Algebra II and science) credit attainment 

• Advanced Placement (AP) credit attainment 

• Dual enrollment credit attainment (measured by college credits accrued) 

• CTE GPA 

We gathered yearly data on these measures over four years, as well as collecting student course- 
taking records and withdrawal information. To determine the contribution of POS on each 
outcome, we used an ordinary least squares regression model. Our analytic models used common 
background characteristics to analyze the POS completion effect on the outcomes of interest. 
Most of these background characteristics came from eighth-grade baseline data provided by the 
district research offices, with exceptions for South District as noted above in Sampling 
Procedures. For regressions run on overall GPA, AP credits earned, and college credits accrued, 
we included the same predictors used in the structural equations models: 

• Gender (dummy coded with female as the omitted variable) 

• Race/ethnicity (dummy coded with White as the omitted variable) 

• Special education (dummy coded with non-special education as the omitted variable) 

• Limited English Proficiency (dummy coded with non-LEP as the omitted variable) 

• Poverty (school indicator for F/RL eligibility, dummy coded with non-eligibility as the 
omitted variable). 

• Eighth grade achievement (standardized academic tests for math and reading) 

• POS effects (dummy coded with the all noncompleters as the omitted variable or CTE 
concentrators as the omitted variable) 

For the regressions run on STEM credits earned, we included all of the above except for reading 
prior achievement. For the regressions run on CTE GPA, we included all of the above except for 
math and reading prior achievement. For West district, we also included dummy variables for 
two of our three schools to account for any school effect: 

• Azure (dummy coded with Navy as the omitted variable) 

• Sky (dummy coded with Navy as the omitted variable) 

We also examined the relationships between POS and postsecondary aspirations and acceptance 
to a postsecondary institution. Exit surveys given to students in their senior year were the source 
for data regarding post-high school plans, and, for some students, the alignment of these plans 
with their high school course of study or POS. We evaluated postsecondary aspiration data with 
crosstab and chi-square analyses. 

All of the analyses in this study used rigorous procedures and standards in the examination of the 
effect of POS on various high school outcomes (U.S. Department of Education, 2013a). For 
example, to check for multicollinearity, we included in our regression statistics the Variance 
Inflation Factor (VIF). To determine the strength of our findings, we included confidence 
intervals around b of 95% confidence. To correct for multiple comparisons, we used the 
Benjamini-Hochberg method. Effect sizes were computed using Hedges’s g. We performed 
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ANCOVAs on each outcome controlling for the same variables that served as predictors in our 
regressions. We obtained adjusted means and used those means in our effect size calculations. 

Limitations . The data for this study did not come from a random sample from the population of 
U.S. secondary students, thus limiting generalization to districts that resemble those included in 
this study. In addition, for the posthoc multiple regression analyses, we limited the sample to 
students who completed four years in the district. This was necessary in order to give all students 
equal opportunity to complete the POS, but by reducing the sample, some generalizability was 
lost. 


Qualitative Procedures and Analyses 

Qualitative methods can assist in the identification of factors that might explain variations in the 
effects of an intervention. Qualitative analyses complement and help explain the more 
quantitative outcomes. Most social programs, such as education programs, are multifaceted — 
some facets function better in some contexts, and other facets thrive in different contexts. It is 
important to explain why an intervention may work where it seems to work. We conducted 
interviews with stakeholders such as students, teachers, counselors, principals, and business 
partners. We observed POS classes, comparison school CTE classes, and academic classes in 
both intervention and comparison schools. Finally, we conducted student surveys as another 
means of eliciting the student perspective. 

Interviews 

Following protocols for research on human subjects and district input, we sought parental 
pennission for students to participate in interviews. In 2009, we paid for a mass mailing to the 
parents of all sample students in West and East Districts in which we invited their students to 
participate in the study. These invitations, sent to parents in their homes, were written in Spanish 
and English and included an infonned consent document for parents to sign and return. During 
the study, we largely interviewed students whose parents signed and returned this form in 2009. 
Near the end of the study, we decided to target at-risk students for interviews in order to ensure 
that we were capturing a broad range of student perspectives and experiences. We selected 
potential interviewees from the dataset based on our definition of at risk (i.e., eligible for F/RL 
and scoring at or below the 20th percentile on any eighth-grade prior achievement measure) and 
submitted their dummy ID numbers to the district research office, which provided the schools 
with the requisite infonnation to send targeted letters to their parents. South District personnel 
selected the students we interviewed at each school site. 

Interviews were audio-recorded and transcribed. Transcripts were classified by type (e.g., CTE 
teachers, students). Following the case study methods of Yin (2009) and the grounded theory 
approach of Strauss and Corbin (1990, 1998), the contents of the transcripts were coded and 
entered into a qualitative data analysis software package called HyperResearch© version 2.8. 
Coding involved marking the instances of concepts or topics relevant to the study as they 
appeared in the interview or observation data. We identified 98 conceptual codes based on our 
site selection process, original site visits, and the elements of the law that we knew we had to 
identify, and we marked the instances in which they appeared in the data. For instance, one 
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element in the Perkins legislation requires POS to lead to industry-recognized credentials or a 
postsecondary degree. Therefore we had a code for these credentials and degrees, and all 
references to them were coded for later retrieval in query reports. We had to delimit the many 
possible concepts to a manageable set that could help explain how the POS experience differed 
from the “default” high school experience at the comparison schools. 

These codes were then queried within HyperResearch and yielded reports rich with infonnation 
on each code from every type of interviewee and observed class. These reports allowed us to 
group the data into more specific topical and descriptive categories (e.g., specific laws that 
support or threaten POS), which we could reduce to a series of matrices (Miles & Hubennan, 
1984) that aided within- and cross-site analyses and helped to identify possible mediating or 
moderating variables that might help explain the outcomes. The qualitative analysis generated an 
understanding of POS in these contexts that was derived and analyzed “from the ground up,” a 
hallmark of grounded theory (Strauss & Corbin, 1998). Such an analysis process maximized the 
triangulation of as many data sources as possible. 

Classroom Observations 

All of our site visits were planned in advance. We worked with our district contacts and school 
principals to select the dates of our visits, then obtained school master schedules in order to 
outline a robust and well-balanced schedule of interviews and observations. All of our interview 
and classroom observation choices were approved by the principal and/or his or her designee, 
who also ensured that teachers were willing to be interviewed or observed. The protocols we 
observed in our classroom observations are detailed in Appendix B and C. All of the completed 
observation forms were also coded and entered into Hyper Research for later retrieval, thus the 
searchable qualitative corpus contained both interviews and observations. 

Surveys 

Two surveys were administered in West and East Districts during the study: a ninth-grade survey 
and a twelfth-grade exit survey. The ninth-grade survey contained sections on course planning, 
POS, career planning, classes and schoolwork, plans for the future, beliefs and opinions about 
school, and background infonnation (e.g., gender, age, race/ethnicity, parents’ level of 
education). The survey was fonnatted for web-based administration in SurveyMonkey and 
configured so as to direct intervention and comparison students into appropriate questions based 
on their responses to key questions (e.g., a comparison student answering “No” to “Do you have 
or are you in a program of study, career major, academy, or program?” would skip questions on 
POS and instead be routed to a subsequent section). 

Student surveys were administered in a variety of ways in order to work within each district’s 
regulations and capacities. Although in both districts, most intervention students could be found 
at the POS high schools, we faced challenges in reaching comparison group students, who were 
distributed across many other high schools in the district. In West District, some schools had 
perhaps a dozen comparison students, whereas others had up to 100. Similarly, in East District, 
although we had only four comparison high schools, participating comparison students had to be 
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identified in each school and invited to take the survey. South District did not participate in the 
student surveys. 

For West’s ninth-grade survey, we sent letters to principals in which we asked them to invite 
participating students to go online and take the district-administered survey using their ID 
numbers as passwords. Survey results were subsequently matched to the dummy ID system and 
returned to us. In East District, district CTE office personnel visited the five participating schools 
and handed out survey invitation postcards to participating students. These postcards provided a 
survey link and their dummy ID numbers, which they would use as their password to the survey. 
This process was feasible at East because of its smaller sample size. Both ninth-grade surveys 
were conducted on SurveyMonkey. 

The twelfth-grade exit survey was administered in May 2012, shortly before our sample 
graduated from high school. This survey contained a section requesting background information 
in order to match responses to student records, as well as a section on post-high school plans 
(i.e., education, employment, military). This survey was fonnatted in SurveyMonkey and 
configured so as to direct students into appropriate follow-up questions based on their responses 
to key questions. The twelfth-grade exit survey requested contact information from students over 
age 18 so that we could contact them after graduation to follow up on their college and career 
experiences. Because we were requesting confidential identifying infonnation, we worked with a 
third-party contractor, who acted as a privacy firewall, to receive the surveys and remove and 
retain the contact infonnation for subsequent follow-up surveys. 

Students were invited to participate in the twelfth-grade exit surveys in the same ways as the 
ninth-grade surveys in each district (in West, through letters to principals asking them to invite 
participating students; in East, through individualized postcards delivered to participating 
students by district CTE office personnel). We added an incentive to the twelfth-grade exit 
survey as a means of obtaining more viable contact information for future follow-ups. In West, 
because we were beholden to the good will of high school principals to encourage participation, 
incentives were for the principals: We offered $200 in Walmart gift cards to the ten schools that 
obtained 80% or higher participation rates in the survey. In fact, 12 high schools beat that 
response rate, and we held a drawing to select the 10 winning schools. In East, CTE office 
personnel felt that a student incentive would be more successful. Prominently displayed on 
students’ invitation postcards was a picture of a Kindle Fire, noting that all respondents to the 
survey would be entered in a drawing for it. 

Survey analysis involved crosstabulating the responses by intervention status and performing 
chi-square analyses to detennine the significance of the differences between intervention and 
comparison group responses. Crosstabs were also run on any baseline differences between 
survey respondents and nonrespondents as well as differences between intervention respondents 
and comparison respondents. These results of these sample comparisons are found in Appendix 
D. 
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CHAPTER THREE: West District 

POS as a tool for improving student achievement and economic development 

Abstract 

West District is an urban district that experienced a large population increase prior to the recent 
economic recession. To respond to this growth, new high schools were built and integrated into 
the district’s school choice program. The impetus behind school choice was to improve student 
achievement, promote diversity, and create an awareness of career opportunities relative to POS 
and magnet themes. 

The school cultures at the three participating POS high schools fostered environments where 
students could both focus on their chosen POS and excel academically. The unique aspects of 
those school cultures are described here. 

We tested an instrumental variable model of the effects of POS enrollment and CTE credit 
earning on GPA and graduation. Our results indicate that enrollment in POS high schools 
improved students’ probability of graduation by 1 1.3% and that increased CTE credit earning 
fully explained this effect. This analysis also showed non-significant effects of POS enrollment 
and CTE credit earning on overall high school GPA. These findings suggest that enrollment in 
West’s POS schools benefited students in terms of retention, at no cost to their achievement. 

At West District, 49.5% of the intervention group completed a POS. We examined policy- 
relevant questions using a second statistical approach to compare the POS completers to students 
who completed a CTE concentration, and to the rest of the sample. Posthoc regression analyses 
were employed to assess the effect of completing a POS on high school achievement: We found 
that compared to the rest of the sample, POS completers were more likely to 1) have a higher 
overall GPA, 2) have a higher CTE GPA, and 3) earn more STEM credits, but completing a POS 
did not affect the likelihood of earning AP credits. The model for dual enrollment predicted a 
higher number of college credits accrued by the rest of the sample, but the results indicated a 
weak fit to the data and suggested that important factors in dual enrollment participation were 
not captured in the model. 

In the comparison with CTE concentrators, POS completers were more likely to 1) have a higher 
overall GPA, and 2) earn more STEM credits, but completing a POS did not affect the likelihood 
of earning STEM or AP credits. The model for dual enrollment showed no predictive value, 
perhaps due again to the weak fit of the data to the model. 

The senior exit survey revealed very similar post-high school plans for both students who 
enrolled in POS and the comparison group: the majority of both groups planned to attend a four- 
year college or university full time, and had already been accepted to the school of their choice. 
Intervention students were more likely to continue their high school curricular focus in 
postsecondary education than were those comparison students who were in a specific program in 
high school (i.e., performing arts). Intervention students were also more likely to have been 
accepted into a branch of the military than comparison students. 
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We then present a qualitative analysis of POS implementation in West District, based on the 
elements of the law mandating POS and the subsequent policy guidance framework provided by 
the funding agency (USDE/OVAE, 2010). 

Introduction to West District and the Intervention Schools 

West District introduced POS as a way to improve human capital and promote economic 
development. A district administrator attributed the development of these schools of choice to 
visionary leaders who saw the rapid growth in the district and decided to provide high-quality 
educational options in all parts of the district. Another administrator described the goal as “elite 
schools for all children” as opposed to “schools for elite children.” 

Three of these POS high schools — Navy, Sky, and Azure — participated in this study. Navy had 
recently been opened before the study began, Sky had a long history of delivering CTE, and 
Azure had built a reputation of “academic distinction and technological excellence.” Each school 
delivered POS slightly differently: Navy through a project-based curriculum, Sky through more 
traditional CTE infused with high-level academics, and Azure through an academic- 
technological focus. These differences reflect practices employed in many high schools 
implementing POS across the country. 


Navy High School 

Navy High School was a relatively new, environmentally green, specially designed facility at the 
beginning of the study: It was the first purpose-built POS high school in the district. In 2008-09, 
Navy served 1,314 students drawn from its regional service area; although racially and 
socioeconomically diverse, the school had relatively more female, White, and Asian students and 
fewer Latino, IEP, LEP, and free-lunch eligible students than the district as a whole. Navy’s POS 
were housed in modular units customized with input from the district’s advisory boards for each 
program area. Modules for each POS consisted of hallways dedicated to the program, with 
academic teachers housed in the area instead of a separate academic wing. 

Navy was designed according to “best practice” tenets of contemporary education, including 
learning communities, integrated curricula, project-based learning (PBL), and the expectation 
that students earn postsecondary credits while in high school and subsequently attend college. 
Although the foundation for each program was career-themed, we heard from several 
administrators that graduates did not always choose their POS for their later careers. At Navy, 
this was not considered a failure: Students gained strong academic skills at Navy as well as 
valuable workplace skills that could help them earn money while they pursued the career of their 
choice. 

POS at Navy 

During the study, Navy offered POS in the areas of transportation, biotechnology, construction 
management, culinary, engineering, hospitality, media, health careers, teacher education, and 
early childhood education. Senior internships were available for most POS at Navy, but students 
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were not required to complete one. Career and technical student organizations (CTSOs) were 
very popular, with dozens of students participating in (and winning) SkillsUSA competitions. 

Sky High School 

Sky High School broadened its traditional mission as the district’s fonner career center to focus 
on providing upgraded, intensive academics while maintaining the school’s culture of high 
expectations and 40-year-old reputation for high-quality CTE instruction. In 2009, Sky’s 
physical plant was showing its age — it needed more instructional space, better facilities, a new 
roof, and a plumbing upgrade. In all other regards, Sky was a high performer and very 
competitive. Sky’s success was the reason the district sought to design and build several new 
high schools around career and technical POS. By the end of our data collection period, these 
expansion and renovation plans had come to fruition. 

Sky’s 2008-09 student body of 1,857, drawn from its regional catchment area, was more female 
and Latino compared to the rest of the district; there were also fewer LEP and free-lunch eligible 
students than the rest of the district. 

POS at Sky 

At Sky, most POS course sequences began in the 10th grade, with the ninth grade dedicated 
largely to academic coursework and other requirements. Sky offered a range of traditional and 
technology-focused POS in such areas as 3D animation, architectural engineering, automotive, 
business, computer networking, culinary, film and video, graphic arts, health, and welding. 

Azure Academy 

Azure Academy is a magnet high school of eight wall-to-wall academies: business and finance, 
computer graphic design, computer-assisted drafting and design (CADD), computer science, pre- 
engineering, infonnation technology, legal studies, and systems technology support. In previous 
years, these programs served as a high-tech complement to Sky’s more traditional CTE programs 
before the district overhaul. Azure’s emphasis was always more college than workforce 
preparatory. Azure was recognized as a Blue Ribbon School and members of its faculty have 
received national awards for teaching excellence. As a magnet, Azure drew its 995 students from 
all over the district in 2008-09. The student body was more male, White, and Asian than the 
district as a whole; it had fewer Latino, IEP, LEP, and free-lunch eligible students. 

POS at Azure 

Due to its status as an early adopter and trailblazer for the other POS high schools in the district, 
Azure was placed in the position of having to “retrofit” its programs when Perkins IV passed in 
order to measure up to the latest standards and supporting components of POS that were laid out 
by Perkins IV, the state, and the district. This included better aligning and sequencing their 
courses into four-course pathways, investigating additional industry certifications, and 
establishing stronger ties to community and business and industry partners. Azure’s challenge 
both during and after our study period was to take the programs that made it a forerunner of the 
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district’s POS and reinvigorate them as full POS without affecting its reputation as a high- 
achieving school. 


How School Culture and POS Identity Engaged Students 

During our visits to the intervention schools, we found that all of the POS met the minimum 
requirements of Perkins IV, with the exception of cosmetology at Sky, which lacked a 
postsecondary component. In one way or another, the comparison school CTE programs fell 
short of POS requirements. Some did not offer any postsecondary or credentialing opportunities, 
others were not organized into coherent sequences, and we saw no evidence in our observations 
or examinations of comparison school course guides that academic courses were aligned or 
cross-referenced to CTE. 

Our observations of CTE in West District suggested that the resources and structures supporting 
CTE were more plentiful in the POS high schools than in the comparison schools. This was not 
due to district favoritism; schools received finite resources and had to allocate them according to 
their diverse needs, missions, and student populations. At the POS high schools, this meant 
strongly supporting CTE; the comprehensive high schools had a wider range of competing 
demands on their funding. As a result, we saw less emphasis on purposeful, required CTE course 
sequencing and career preparation at the comparison schools compared to the intervention 
schools. 

We identified other practices at the West District POS high schools that aided in POS 
development and implementation, such as using innovative teaching techniques that included 
applying academic learning to career contexts, and developing a sense of identity around POS. 
Many of these practices, which fonn a vital part of the POS high schools’ cultures, were not 
evident in our visits to the comparison schools. This does not mean they did not exist there; 
however, comparison school personnel did not indicate that these practices occurred at their 
schools. Some of the practices that foster these different school cultures are described below. 

Navy: The Cutting Edge 

Integrated technology. A recently-built school, Navy featured modern, up-to-date facilities, full 
online connectivity, and innovative project-based teaching and learning strategies. For example, 
its culinary and hospitality facilities, including a full-sized banquet hall, industrial kitchens, and 
demonstration classroom, were described by its chef-instructors as better than the community 
college’s, which uses the school’s facilities for evening classes. Classes we observed used digital 
cameras to record lessons to be broadcast later as podcasts; these were seen as helpful to all 
students, but especially for those IEP students who needed to review lessons several times for 
full understanding. 

Project-based learning (PBL). PBL was a guiding principle at Navy. PBL combines academic 
and technical content knowledge and hands-on applications of that knowledge, culminating in an 
annual multi-day open house of themed projects. During one year of our observations, the theme 
of this open house was space exploration and Earth conservation. The event featured poster 
tables, multimedia presentations, technology demonstrations, and mock trials. Transportation 
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students designed space rovers, biotechnology students cultivated a hydroponics laboratory that 
simulated various natural environments, and health careers students explored the impacts of 
space travel on human health and fitness. Some students took on methodologically sophisticated 
science projects; one student studied the effects on bacteria of a common antibacterial agent 
found in many personal care products. Such students worked with adult experts outside of the 
school, and one student had secured herself an internship with a prestigious medical clinic in the 
area. 

Support. Navy’s culture nurtured both students and teachers. Notable was its advocacy period, 
during which students engaged in one-on-one mentoring and advising with a teacher who 
remained their mentor throughout their four years of high school. Teachers received support on a 
variety of fronts, including the principal’s open-door policy and commitment to supporting 
curricular innovation and on-going professional development. 

Sky: A Tradition of Pride and Success 

Familial culture. As one of the older high schools in the district, Sky enjoyed a familial sense of 
continuity, shared history, and communal values that had been passed on to succeeding cohorts 
of students. The school’s values were communicated to new students through a combination of 
establishing and reinforcing mechanisms and customs that encouraged students to take pride in 
the school’s appearance and their own accomplishments. Students neatly stacked up their chairs 
after the end of the lunch period; a welcome luncheon was served to the new freshmen by the 
senior class; and students and teachers were overtly praised and rewarded for their 
accomplishments. Sky actively promoted itself as a place of excellence and achievement: It 
distributed infonnation sheets touting its low dropout rate and high attendance and graduation 
rates. The school staff we interviewed trumpeted the school’s many AP classes, strong pass rates 
on the state achievement exams, healthy college-going numbers, and reputation for safety. 

Hands-on learning. Students engaged in hands-on learning at Sky. One health careers student 
stated that she was happy at Sky “because usually in regular high school, they don’t have any 
hands on. You’re just listening and writing on paper.” She described how her health careers POS 
teachers brought in “biology, chemistry, anatomy all into it to understand the body and functions 
too. And math- they use it too, for calculations of blood pressure.” 

Soft skills and professional standards. A required ninth-grade class at Sky taught soft skills, 
career exploration, and standards of professional behavior. In this class, students prepared 
resumes, charted out their four-year graduation plans, and participated in mock interviews with 
community business partners. During our observations of this class, students were being 
evaluated on the professionalism of their dress. Sky’s Career Center served as a model for the 
rest of the district: It included a job bank, cooperative education agreements, and computers with 
access to career interest inventories, college and career information, and industry assessments. 

Azure: The Technology Flagship 

High achievement. As one of the district’s highest achieving schools, Azure had a reputation to 
maintain. We heard this from district personnel and most interviewees at Azure, including the 
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students. Azure’s focus is academic achievement within career-themed programs. For example, 
the school hosted popular CTSOs like Future Business Leaders of America (FBLA). A business 
teacher we interviewed stressed the importance of FBLA in imparting dress, networking, and 
written and oral communication skills to students in addition to providing opportunities for 
business-related field trips and leadership development activities. A student described the FBLA- 
sponsored management and decision-making competitions she took part in that required 
teamwork to resolve an issue and then to present the resolution to judges. The connection 
between high academic achievement and careers seemed clear to the teachers we spoke with: “I 
always tell students, when you get into the workforce, employers want someone who’s well 
rounded.” This CTE teacher exhorted her students to excel in both academics and CTE: “Content 
is the foundation of the [POS] program area. The math, the English, the history. It all works 
together.” 

Self-awareness. One Azure counselor reported that the school was full of self-described “nerds.” 
The students we spoke with described their strong focus on schoolwork and goals. One called 
Azure “a good school for going to college,” and another told us that she had applied to Azure 
because she wanted higher standards. She believed that students back at her zoned high school 
“don’t think about their future as much as they do here.” 

Instrumental Variable Sample Description 

We examined data for 2,004 secondary students; 1,175 enrolled in three POS high schools in 
ninth grade; 829 attended other schools. Table 3.1 displays descriptive statistics for the West 
District imputed sample. 


Missing Data 

We carried out a missing data analysis and observed that the percentage of data missing for the 
variables ranged from 0.0 to 3 1 .3%, with an average of 7.6%. We rejected the assumption that 
the data were missing completely at random (MCAR) on the basis of mean differences in eighth- 
grade standardized test scores between those who were and were not missing GPA data for later 
years. Thus, we found it necessary to make the more relaxed assumption of missing at random 
(MAR), used multiple imputation to handle the missing data (Little & Rubin, 2002), and 
generated five imputed sets using NORM 2.03 (Schafer, 1997). 

Instrumental Variable Estimation 

We tested our hypothesized model across the imputed data sets and observed that it was over- 
identified with 13 degrees of freedom and fit the data well (e.g., f = 20.75, CFI = .99, TLI = .97, 
and RMSEA of .02). We tested the model using a single imputed set to obtain a f test and 
modification indices. We observed a statistically significant global test statistic {p = .02), 
suggesting that the model did not fit the data exactly (see Bollen, 1987, pp. 263 - 269). The 
observed modification indices and standardized residuals suggested that number of CTE credits 
earned should be regressed on gender. We added this specification and observed that the 
resulting model was over-identified with 12 degrees of freedom and fit to the data was excellent 
(e.g., 'i = 8.75, CFI = 1.00, TLI = 1.00, and RMSEA = .00). We tested the model using 
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Table 3.1 


Descriptive statistics for imputed sample, West District 



(A =2,004) 

% 

Overage 



No 

1,937 

96.7 

Yes 

67 

3.3 

Gender 



Male 

847 

42.3 

Female 

1,157 

57.7 

Race/Ethnicity 



Black 

194 

9.7 

White 

804 

40.1 

Latino 

678 

33.8 

Asian 

313 

15.6 

Other 

15 

0.7 

F/RL eligible 

558 

27.8 

Limited English Proficient (LEP) 

50 

2.5 

Has Individual Education Plan (IEP) 

82 

4.1 

Graduated on time (2012) 

1,803 

90.0 

Attrition 

105 

5.2 


Mean 

SD 

Discipline incidents 

0.51 

1.27 

Grade 8 pre-test math scale score 3 

369.51 

77.02 

Grade 8 pre-test reading scale score 3 

345.99 

54.88 

Grade 8 pre-test science scale score 3 

366.64 

64.49 

Number of CTE credits earned 3 

5.15 

2.71 


Note. F/RL = free/reduced price lunch program. 
a Variables were grand mean centered. 


a single imputed set and observed a non-significant global test statistic (p = .46), suggesting that 
we should not reject the hypothesis that the model fit the data exactly. This result implies that the 
instrumental variable model assumptions of nil direct effects of enrollment in POS schools on 
GPA and graduation were consistent with the data. We also observed that the covariances of 
number of CTE credits earned with GPA and graduation were covs = .34 and .61 (ps < .001), 
respectively. These significant residual covariances suggested that these variables shared 
variance attributable to omitted variables and that our instrumental variable was necessary to 
recover consistent estimates (see Antonakis et ah, 2010). 

For this model, enrollment in a POS high school appeared to increase the number of CTE credits 
students took by an average of 2.82 credits (see Figure 3.1). Controlling for background 
variables, the indirect effect of enrollment in POS high schools on GPA was not significant (p = 
.959), while its indirect effect on graduation was b = .34 (p < .001). The latter effect is a probit 
coefficient and can be interpreted as indicating that holding the background variables constant, 
enrolling in a POS high school increased students’ probability of graduation by an average of 
1 1.31% (see Figure 3.2). The direct effect of CTE credits on GPA was not significant (p = .960), 

43 



while its direct effect on graduation was b = .12 (p < .001). The latter suggests that each 
additional CTE credit earned by students increased their probability of graduation by 4%. Taken 
together, these results indicate that (1) students who enrolled in POS high schools were 1 1.31% 
more likely to graduate than students who enrolled elsewhere, and (2) this effect is explained by 
their increased earning of CTE credits. 

Figure 3.1 

Final model for West District 



0.19 

Note: Significant unstandardized estimates are displayed. Covariates are omitted for conceptual clarity. 

In addition to the effects described above, several covariates were significant predictors of 
number of CTE credits earned, GPA, and graduation. Gender (i.e., being male), F/RL, LEP, 
special education, discipline events, Black, and Latino had significant and small negative effects 
on GPA, while Asian and Grade 8 science and reading scores had a significant and small positive 
effect on GPA (all ps < .05). Grade 8 math score had a significant and moderate positive effect 
on GPA (p < .001). Age, F/RL, and discipline occurrences had significant and small negative 
effects on graduation, while Asian and Latino had significant and small positive effects on 
graduation (ps < .05). Finally, gender had a small but significant positive effect on number of 
CTE credits earned (p < .001). Note that each of these effects should be interpreted as the effect 
of a particular predictor, holding all other predictor variables constant, or within levels of the 
other modeled predictors. For a complete description of the estimates for West District, see 
Appendix A. 
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Figure 3.2 

Probability of graduation, West District 

1.00 -r 


a -95 

o 



Note: Estimated probabilities holding constant age; gender; race/ethnicity; free lunch status; LEP; IEP; Grade 8 
science, math, and reading scores (grand mean centered); and discipline incidents. 

Assessing the Effect of Completing a POS on High School Achievement 

As noted earlier (see Method), we employed two statistical approaches to analyze the data. Our 
second approach employed multiple regression analyses, using a different sample — POS 
completers, CTE concentrators, and the rest of the sample (All Others). For these analyses, we 
excluded students who (1) withdrew or were missing course data and (2) were missing baseline 
achievement data. We were consistent with this even with the CTE GPA analysis (that did not 
control for baseline achievement) so that the regression sample would match the descriptive 
statistics to the extent possible. Of course, the sample for the CTE GPA analysis is further 
reduced to those who actually took a CTE course among the students in the All Others group. 
Table 3.2 provides descriptive statistics for the posthoc regression sample. 

Our analyses found that of the 1,175 intervention students at West, 582 (49.5%) completed a 
POS. We also identified the number of students who earned credits in the outcome measures 
outlined in the Method chapter. These included measures of academic rigor such as earning 
STEM credits (i.e., credits in science and higher math courses, which we defined as courses 
above Algebra II), and AP credits, as well as a transition measure; accruing college credits 
through dual enrollment. Finally, we included a measure of overall GPA because of its predictive 
value for college success, and a CTE GPA measure in order to gage technical achievement. All 
of these measures have high policy relevance; A major impetus behind POS is for CTE students 
to take more rigorous coursework that leads to postsecondary education. Descriptive results of 
student course-taking are shown in Table 3.3. 
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Table 3.2 


Descriptive statistics, posthoc sample, West District 



Total 

(n = 1,368) 

POS Completers 
(n = 545) 

All Others 
(n = 823) 

CTE 

Concentrators 
{n =130) 

POS 

Completers 
Compared 
to All 
Others 

POS 

Completers 
Compared to 
CTE 

Concentrators 

Characteristics 







Male 

41.6 

51.0 

35.4 

44.6 



Black 

9.9 

10.5 

9.6 

11.5 



White 

38.7 

40.9 

37.3 

38.5 



Latino 

34.4 

30.8 

36.8 

40.0 



Asian 

16.0 

16.7 

15.6 

8.5 



Other/Multiracial 

0.9 

1.1 

0.7 

1.5 



F/RL eligible 

27.9 

26.4 

28.8 

34.6 



LEP 

2.1 

1.1 

2.8 

1.5 

* 


IEP 

4.1 

2.4 

5.2 

6.9 

** 

** 

Grade 8 pre-test reading 

347.87 (54.66) 

348.99 (51.48) 

347.13 (56.68) 

336.45 (55.26) 



Grade 8 pre-test math 

370.80 (76.60) 

372.92 (73.65) 

369.40 (78.54) 

357.89 (87.10) 



Discipline events 







None 

76.3 

75.8 

76.7 

68.5 



1-2 events 

18.3 

17.6 

18.7 

24.6 



3+ events 

5.4 

6.6 

4.6 

7.0 




Note. Excludes students who withdrew or were missing course data. Standard deviations of continuous variables are included in parentheses. 
*p < .05, **p < .01. 
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Table 3.3 


Descriptive outcome statistics, posthoc sample, West District 



Total 

(n = 1,368) 

POS Completers 
(n = 545) 

All Others 
in = 823) 

CTE 

Concentrators 
(n = 130) 

POS 

Completers 
Compared 
to All 
Others 

POS 

Completers 
Compared to 
CTE 

Concentrators 

Percent Earning Outcome Credits 






STEM 

100.0 

100.0 

100.0 

100.0 



AP 

49.2 

52.3 

47.1 

36.9 


** 

College (accrued) 

5.6 

0.4 

9.1 

1.5 

*** 


CTE 

88.7 

100.0 

81.3 

100.0 

*** 


Outcome Means 

M{SD ) 

M{SD ) 

M{SD ) 

M{SD ) 



Overall GPA 

3.31 (0.61) 

3.38 (0.51) 

3.27 (0.51) 

3.18(0.60) 


** 

STEM credits earned 

5.20 (2.01) 

5.26 (1.77) 

5.16(2.15) 

4.60(1.49) 


*** 

AP credits earned 

1.63 (2.37) 

1.56 (2.09) 

1.67 (2.54) 

1.09(1.94) 


* 

College credits accrued 

1.01 (5.54) 

0.07(1.13) 

1.63 (7.02) 

0.08 (0.63) 

*** 


CTE GPA 

3.32 (0.64) 

3.41 (0.43) 

3.24 (0.76) 

3.30 (0.61) 


* 


n = 1,215 


n = 670 





Note. Excludes students who withdrew or were missing course data. STEM credits equals credits in science and higher math courses (i.e., above Algebra II). 
College credits accrued refers to the potential college credits associated with dual enrollment participation in high school. 

* 'M * *** 

p < .05, p < .01, < .001. 
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Following are the results of our posthoc analyses of high school achievement, controlling for 
background characteristics (full regression tables are found in Appendix A). Comparing POS 
completers to the rest of the sample, the results for overall GPA, STEM credits earned, and CTE 
GPA are positive and statistically significant (see Table 3.4). Of the four statistically significant 
findings, two (overall GPA, CTE GPA) have effect sizes of 0.25 or larger, making them 
substantively important as well. In contrast, the college credits accrued (dual enrollment) finding 
is negatively associated with completing a POS, and contributes relatively little to the variance 
explained of the difference between completing and not completing a POS in terms of accruing 
college credits. 

Table 3.4 

Regression results, POS completers versus all others, West District 

West District POS Completers Versus All Others 


Outcome 

n 

Adjusted 
R 2 

b 

SEb 

fi 

P 

95% C7 of b 
lower upper 

ES 

Overall GPA 

1,368 

0.385 

All 

.029 

.142 

.000 

.121 

.232 

0.30 

STEM 

credits 

earned 

1,368 

0.268 

21b 

.102 

.067 

.007 

.075 

All 

0.14 

AP credits 
earned 

1,368 

0.257 

-.146 

.122 

-.030 

.232 

-.385 

.093 

-0.07 

College 

credits 

1,368 

0.021 

-1.667 

.327 

-.147 

.000 

-2.308 

-1.025 

-0.30 

accrued 

CTE GPA 

1,215 

0.085 

.155 

.038 

.121 

.000 

.082 

.229 

0.27 


Note. All coefficients rounded to two digits. Full regression tables are found in Appendix A. CI= Confidence 
Interval. ES = effect size (Hedges’s g) 


Table 3.5 shows the results of the comparison of POS completers and CTE concentrators. The 
results for overall GPA and STEM credits earned are positive, statistically significant, and both 
have effect sizes larger than of 0.25, making them substantively important. The dual enrollment 
finding is not significant; however, the poor fit of the model to the data make the finding 
questionable. 

Looking more specifically at the covariates in the full regression tables (cf. Appendix A), 
students attending Navy were significantly more likely to have a higher overall GPA than 
students attending Azure ip = .000 for both All Others and CTE concentrator models) or Sky (p 
= .000 for All Others model and p = .013 for CTE concentrator model; see Appendix A for all 
covariates). This suggests a school effect at Navy that could be attributed to aspects of Navy’s 
structure, curriculum, and culture: Navy assigned academic teachers to team with specific POS, 
and they were co-located in the same building wings for easier collaboration. This collaboration 
was expected within their problem-based learning curriculum. There were also opportunities to 
participate in work-based learning experiences at Navy — the operational child care facility 
onsite, the active culinary and hospitality programs that often hosted community and district 
events, and internship opportunities. All of these could have engaged students and shown them 
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how achievement in both academic and technical courses was necessary for the careers they 
wished to pursue. No other school effect at West was as clear cut as the overall GPA outcomes. 


Table 3.5 

Regression results, POS completers versus CTE concentrators, West District 

West District POS Completers Versus CTE Concentrators 


Outcome 

n 

Adjusted 

R 2 

b 

SEb 

fi 

P 

95% Cl of b 
lower upper 

ES 

Overall GPA 

STEM 

credits 

675 

0.391 

.152 

.042 

.112 

.000 

.071 

.234 

0.30 

earned 

AP credits 

675 

0.313 

.541 

.143 

.123 

.000 

.261 

.821 

0.31 

earned 

College 

675 

0.332 

.178 

.168 

.034 

.290 

-.152 

.508 

0.08 

credits 

accrued 

675 

-0.011 

-.016 

.105 

-.006 

.875 

-.223 

.190 

-0.02 

CTE GPA 

675 

0.066 

.078 

.045 

.065 

.085 

-.011 

.168 

0.19 


Note. All coefficients rounded to two digits. Full regression tables are found in Appendix A. Cl = Confidence 
Interval. ES = effect size (Hedges’s g) 


Senior Exit Survey Findings 

A senior exit survey was conducted in May 2012, including all seniors in our original sample 
(see Method). Response rates were not high: The intervention student response rate was 38.8 
percent and the comparison student response rate was 39.2 percent — not a significant difference 
(p = .858). 


The comparison group’s survey respondents were not significantly different from the comparison 
nonrespondents; among the intervention group students, significantly more female and more LEP 
students did not take the survey (see Appendix D ). Appendix D also shows differences between 
intervention and comparison survey respondents. We found that significantly more males and 
Black intervention students, and significantly fewer students with an IEP took the survey than in 
the comparison group. All of these differences should be taken into account when interpreting 
the survey results. 

We found that the students who responded to the survey were very similar in their post -high 
school plans, regardless of whether they were in the intervention or comparison group (see Table 
3.6). Most had been accepted (over three fourths of the sample) and planned to attend a four- year 
college or university (over 68% of the sample) full time (over 91% of the sample) in Fall 2012. 
There were no significant differences in their work plans either: most respondents planned to 
work part time, and those jobs were not related to their high school program. Only a small 
percentage planned to join the military, but intervention students were more likely to do so than 
comparison students. 
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We asked students whether their postsecondary studies would be related to their high school 
program. Of those students who indicated that they were in a high school program (defined in the 
survey as “major, program of study, career major, career pathway, career academy, International 
Baccalaureate, etc.”), 60 percent of the intervention students indicated that their post-high school 
program would related to their high school program. By contrast, less than half (48%) of 
comparison students indicated that their post-high school studies would relate to their high 
school program. This suggests that participating in a high school program helped clarify 
intervention students’ post-high school planning. 

Table 3.6 


Student exit survey responses, West District 


Survey Items 

Intervention 
(TV = 456) 
(%) 

Comparison 
(N= 325) 
(%) 

Do you plan on working immediately after graduation? 

Yes 

62.5 

61.2 

Will you be employed full time? 

Yes 

34.0 

43.7 

Is this job related to your high school program? 

Yes 

25.6 

8.0 

No 

67.4 

43.2 

Not in a program 

7.0 

48.7 

Do you plan to attend a four- year college or university, 
two-year community college, or trade/ technical school 
for the Fall 2012 semester? 

Yes 

91.7 

88.9 

Four- year 

68.7 

72.0 

Two-year 

27.5 

24.2 

Trade/technical 

3.1 

2.8 

Other 

0.7 

1.0 

Have you been accepted to this school? 

Yes 

81.6 

76.8 

Will you be a full-time student? 

Yes 

91.4 

94.1 

Will your studies at this school relate to your high 
school program? 

Yes 

56.9 

27 o*** 

No 

37.8 

28.7* 

Not in a program 

5.3 

44 2 *** 

Have you been accepted into a branch of the military? 

3.1 

0.9* 


Note. Due to missing data, percentages may not add up to 100. Significance levels are based on chi-square analyses 
comparing intervention and comparison groups. 

*p < .05. **p < .01. ***p < .001. 
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Adherence to the Components of POS 


At the beginning of our study, West District’s POS, constructed as non-duplicative course 
sequences aligning secondary and postsecondary content in programs typically requiring three to 
four years of progressively more challenging coursework, were largely delivered in purpose- 
built, regionally located POS high schools. Students meeting basic eligibility requirements 
applied to these choice programs through a district lottery. POS were also offered in magnet 
programs housed in some of the district’s comprehensive high schools. By the end of our data 
collection activities, West was in the process of developing and implementing POS in all of its 
comprehensive high schools offering CTE. Block scheduling greatly facilitated POS 
implementation in the POS high schools; the comprehensives, which did not offer such 
scheduling, needed more flexibility. Because districts and other local education agencies in this 
state tended to exert strong local control, West was able to dictate its own policy with respect to 
the expansion of POS in all of its high schools. To a certain extent, West’s CTE programs and 
intensive POS served as exemplars to the rest of the state. Indeed, during our study, the state 
began standardizing its CTE programming, in some cases adopting the work completed by West. 

Nearly a dozen advisory committees comprised of secondary and postsecondary faculty and 
business and industry partners designed West’s POS to align with postsecondary and industry 
content and standards. Advisory committees generally met monthly to review and update 
programs, and committee members were nearly universally praised by study participants as 
contributing significant expertise and considerable material and financial resources to district 
POS. Of these partners, West worked particularly closely with West Community College (WCC) 
to articulate nearly every CTE program offered by the college with similar programs in its high 
schools. As such, opportunities to earn college credits, through these articulations or a more 
limited dual enrollment program, were abundant in this district. Overall, but to varying degrees 
of intensity, West’s POS incorporated project-based learning, curriculum integration, CTSO 
leadership activities, intensive career guidance and exploration, opportunities to participate in 
work-based learning and earn industry-recognized credentials, industry-quality technology, and 
embedded soft skills instruction. Administrators, teachers, and counselors reported receiving 
abundant professional development support from the district; some of this development was 
offered by WCC. Although technical skills assessments were not in place at the beginning of our 
study, by 2010, the district was working with the state to align every POS with an industry- 
recognized credential or third-party assessment, where such were available. 

Incorporate and Align Secondary and Postsecondary Education Elements 

Working in conjunction with WCC and/or state university faculty, West District developed the 
POS curricula at its POS high schools to prepare students to pursue further education at WCC or 
one of the state’s four-year universities. 4 Indeed, the secondary POS curriculum development 
process drew upon state higher education standards so that developers could visualize not only 
the postsecondary end point of a given POS, but also which components of a program could be 
offered in high school for college credit. In some programs, the senior-year POS course was 
deemed the equivalent of a postsecondary course (e.g., an automotive course at WCC, an 


4 If a program did not have a postsecondary component, such as Sky’s cosmetology program, we did not consider it 
a POS for the purposes of this study. 
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engineering or teacher education course at the state university). Through its articulation 
agreements with WCC and the state universities, West could offer such courses for college 
credit, thus eliminating the need for students to take them at the college level. 

We were able to observe the process of secondary-to-postsecondary curriculum development and 
alignment in action at an advisory board meeting for West District’s construction programs. 
Present were construction technology teachers from West high schools, a number of district CTE 
personnel, a WCC department chair, and several business and industry partners. The goal of the 
meeting was to ensure that the high school curriculum would continue to align with WCC’s 
construction programs; at that time, WCC was in the process of launching new degree and 
certificate programs in sustainable construction. This meeting allowed the entire advisory board 
to not only learn more about these pathways, but also plan how West high schools could develop 
curricular modules leading to them. 

When West District made changes to its own CTE curricula or course titles, such changes had to 
be approved by WCC in order to maintain curricular alignment. This process sometimes required 
industry input. As one WCC administrator noted, the business of aligning secondary and 
postsecondary curricula resembled a “food chain” in which industry — at the top — told 
postsecondary educators how to structure their programs; postsecondary faculty then worked 
with the district and secondary-level teachers to redesign their programs to make sure that 
students met WCC entrance requirements. The same WCC administrator described the 
relationships between WCC faculty and high school teachers as varying across program areas. 
Some programs maintained a regular schedule of outreach to their high school counterparts; 
others developed program leadership meetings or daylong retreats to examine course content, 
methods, and evaluation processes; still others were more haphazard in the frequency and 
intentionality of encounters across institutions. 

West District POS had other postsecondary components besides their curricula. Some used 
college textbooks; others featured teachers who also held adjunct teaching positions in the same 
subject area at WCC. Many high school teachers told us that they pursued additional education 
and professional development at WCC, where industry partners also said that they went for their 
training. 

Secondary and postsecondary POS also shared facilities. In some cases, high school facilities 
met industry standards better than those at the college; in others, the reverse was true. To the 
extent possible, West District and WCC made every effort to share. One notable example was a 
heating, ventilation, and air conditioning (HVAC) program that suffered from low enrollment at 
several high schools despite a high demand for workers in the area. The program’s advisory 
board convinced the board of education to consolidate all district training at one site — Fuchsia, a 
comprehensive high school located next door to a WCC satellite campus that housed a nationally 
accredited HVAC training center. High school and WCC faculty worked together to write the 
state standards for the program, which was accredited by the Partnership for Air-Conditioning, 
Heating, Refrigeration Accreditation (PAHRA). High school students in the program took their 
regular courses at the high school, then walked across the parking lot to receive their career 
instruction at the WCC training facility, which served adults at other times of the day. The 
district’s articulation agreement with WCC allowed West students who completed the program 
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to earn as many as 13 college credits. Students could also take their general education 
requirements through WCC, making it possible to graduate from high school with a diploma and 
a completed WCC HVAC certificate. Although Fuchsia’s principal strongly supported the 
program, dwindling enrollment numbers and the retirement of its acclaimed high school 
instructor in the fourth year of our study ultimately spelled its demise, despite WCC’s best 
efforts to provide an interim instructor and incentives for currently enrolled students to complete 
their pathways. 

A WCC administrator believed that West’s heavy investment in POS high schools reflected its 
desire to incorporate postsecondary education into the high school experience. Because West 
students enrolled in high school POS voluntarily, such programs were more like college. As he 
noted, at the POS schools, faculty could “focus on industry requirements, industry outcomes, 
career focus, that kind of stuff. Makes an entirely different structure there.” By contrast, when 
students were present only because they had to be, which this administrator felt was true at the 
comprehensive high schools, CTE programs could not coordinate well with postsecondary, 
“because [the schools’] focus is classroom management or attendance, not career outcomes.” 

At the state level, administrators recognized the need for representatives of secondary and 
postsecondary education to collaborate. Postsecondary faculty offered guidance on connecting 
programs during the state’s technical assistance workshops on POS. And a state department of 
education official reported that his agency was working with the state higher education 
commission to make sure that POS spanning both systems were a priority in their work and 
future grantwriting efforts. 

Include Academic and CTE Content in a Coordinated, 

N on-Duplicative Progression of Courses 

West worked to design and implement non-duplicative POS course sequences that aligned 
secondary and postsecondary content in programs requiring as few as two to as many as four 
years of high school coursework before leading to postsecondary education. Both the state and 
the district recognized the need to offer flexible POS course sequences, particularly for those 
comprehensive high schools that followed non-block schedules or offered fewer CTE options. 
The state’s CTE director noted that many rural districts had limited resources, fewer connections 
with industry, and were sometimes located far from a college campus. As he noted, in such 
districts, schools could offer the first two levels of an articulated POS sequence, with the third to 
be taken at the postsecondary level (e.g., Automotive I and II in high school, and Automotive III 
at a community college). He believed that although three-year sequences were more desirable, 
two-year sequences (especially in the comprehensives) still held the promise of rigor. 

With its more abundant resources and public commitment to building and sustaining the POS 
high schools, West District designed its aligned course sequences with considerable input from 
about a dozen advisory boards comprised of secondary, postsecondary, and industry partners. 
The district’s CTE director described this as offering principals a POS “menu” of two- to four- 
year programs — although the POS we observed all spanned three to four years in length — 
consisting of entry, mid-level, and tenninal (i.e., capstone) courses coupled with industry 
credentials and ending with state competency certificates. After taking a “painful” look at its 


53 



curricula and enrollment numbers, the district eliminated resource-draining singleton courses and 
programs with low enrollments and streamlined pathways into each of the sector strategy-driven 
areas. By the fourth year of the study, we saw evidence of the district’s effort to roll out aligned 
sequences in a number of magnet programs at its comprehensive high schools. 

One retired district administrator who was instrumental in the early design and implementation 
of POS told us how they had aligned and sequenced academically and technically rigorous POS 
that would incorporate postsecondary and industry standards and hands-on, experiential learning 
and lead to further education and high-skill, high- wage jobs in high-demand areas. District 
personnel used a “backward assessment” process that began with their goal — students graduating 
with viable, marketable diplomas, meaningful credentials, and varied skills that would allow 
them to adapt to a changing job market — and constructed the curriculum backwards from there: 

We tried to start. . . where we thought students might finish and then go backwards and 
say. . . what do we need to put in place to maximize the opportunity for every student? 
They’re not all going to decide at the same point that they want to go on [to college], but 
we want to make sure that they understand that that’s clearly where we want them to go 
and give them every opportunity to access that. . . I think it’s really made a difference. It 
really helps kids focus and it gets them excited about taking more rigorous coursework 
when they can see the application. 

The district was aided in this work by WCC, which had committed to offering high school 
students a quick route into its professional programs. The college employed a full-time 
coordinator who worked with each of the district’s high schools to educate principals, program 
chairs, teachers, and guidance counselors about available dual credit opportunities. It is worth 
noting that WCC’s investments in creating aligned secondary-postsecondary program sequences 
did not always lead to successful outcomes for the college. A number of WCC faculty and staff 
we interviewed reported that students were not always well-prepared to take and pass mid-level 
courses. Some college faculty also questioned the rigor of some aligned high school programs; in 
culinary, for example, the department chair spoke of the disconnect between high school 
programs offering FACS-based (Family and Consumer Sciences) food and nutrition courses and 
those offering industry-driven cooking and baking courses. 

N on-Duplicative Sequences at POS Schools 

The hallmark of all of West’s POS sequences was the close alignment of program curricula with 
postsecondary programs, as reflected in the wide availability of college credit from WCC (and in 
some cases from the local four-year university). There was also a strong, shared expectation that 
most students would pursue some form of postsecondary education after graduation. Facilitating 
alignment and sequencing was the purposeful structuring of the district’s POS high schools to 
include block scheduling, project-based learning, curriculum integration, work-based learning, 
and connections with postsecondary and industry partners. Navy provided many examples of 
secondary-postsecondary alignment. All of the teachers and program chairs we interviewed were 
highly articulate regarding the alignment and sequencing of courses in their program areas. POS 
differed in the depth of their alignment with postsecondary, however. Some programs, like 
Navy’s biotechnology program, were developed in concert with faculty at the local university. In 


54 



the case of Navy’s teacher education program, the lead teacher developed the curriculum, which 
included rotations in the school’s on-site kindergarten, based on her own postsecondary 
experiences. She was rebuffed in her efforts to align the program with WCC. The hospitality 
chair described working with WCC, the local four-year university, and the American Hotel and 
Lodging Association (AHLA) to design a standards-driven curriculum built around the 
postsecondary institutions’ needs. A transportation teacher noted that his program offered the 
identical sequence of courses as at WCC — where he used to teach — with the added benefit of 
more time to delve deeply into the content. Annual course selection guides helped students 
construct their schedules by laying out clear program strands, course sequences, related electives, 
and available credit transfer opportunities. 

Non-Duplicative Sequences at Comparison Schools 

West District provided additional POS opportunities in magnet programs located on 
comprehensive high school campuses. But there were otherwise few to no POS sequences in 
evidence at the comprehensive high schools with “regular” CTE. Ruby’s magnet coordinator 
described their EMT/first responder program as the school’s only full POS sequence, which she 
defined as providing opportunities to earn college credit. Although these credit transfer 
opportunities were considered an easy sell to Ruby students, the coordinator believed that there 
were not enough college credit-eligible courses being offered. Students in the school’s aviation 
magnet POS took a four-year pathway of aerospace courses that culminated in a senior capstone 
and an internship at the local university; students were also qualified to test for a private pilot’s 
license. Ruby’s principal was aware that his school faced competition from the district’s POS 
high schools; he perceived an advantage, however, in Ruby’s ability to offer the kinds of 
standard high school experiences — including athletics — not offered at the POS high schools. 

Student perspectives on their course sequences. Over the course of the study, most interviewed 
students demonstrated a strong understanding of their POS majors and course sequences and the 
options available to them at their schools. Students at Sky and Navy, particularly, seemed to 
benefit from opportunities in ninth grade to experiment with other program areas before fully 
committing to their POS. Such experimentation took the form of rotations — by semester or nine- 
week blocks — through different programs. In Navy’s culinary and hospitality programs, for 
example, freshmen spent one semester in each area in order to experience both “front of the 
house” and “back of the house” activities. One Navy senior said that this experience helped him 
choose hospitality over culinary; although his ultimate career goal was not hospitality-focused, 
he believed that the customer service skills he had learned would make him a better future 
business owner. One Sky student reported that her rotation through culinary did not excite her 
nearly as much as health science; she was planning to test for her CNA as a junior and specialize 
in sports medicine as a senior. Overall, most participants acknowledged that once in a program, it 
was nearly impossible to change POS after 10th grade given the increasing complexity and 
specialized content knowledge being taught. However, most students we spoke with said that 
they were not only in their first choice of school, but also in their first choice of program. A 
number of students said that, because seats were limited in their POS of choice, they were 
compelled to select a second or third choice. One Navy senior who was unable to get into the 
school’s culinary program had to select between his second choice, transportation, and early 
childhood education. Happily, transportation presented challenges that this student believed he 
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simply would not have had at his zoned high school: 

After the first two weeks of being here, ... I felt like [it] was challenging me enough to. . . 
actually feel like I was progressing in my education. . . I thought it was just going to be 
learning about cars and what kind of cars there are and what kind of fuels there are. I 
didn’t know that we were actually going to build our own. So that kind of surprised me in 
my freshman year when we made our first mechanical mousetrap cars. Instead of using 
gasoline or solar power, we used [a] mousetrap with a string. 

Other students had a less pleasant experience. By Year 4 of the study, budget cuts forced the 
closure of a number of Sky’s programs, including business administration, accounting, and 
welding. Students in those programs were compelled to choose from a limited number of 
programs with available seats. A commercial construction instructor noted that some of these 
students were shunted into his classes against their will, and some self-sabotaged to escape. 

Offer the Opportunity, Where Appropriate, for 
Secondary Students to Acquire Postsecondary Credits 


Dual Enrollment 

Compared to the robust articulation between institutions, a small dual enrollment program 
existed between West District and WCC — only about one quarter of West high schools 
participated because the courses had to be taught at the high schools by West District high school 
teachers who were certified to teach college-level courses. Not all teachers wanted to or were 
qualified to teach such courses. In the high schools that did offer dual enrollment, most of the 
courses taken were general education courses, which were recorded on college transcripts. Some 
CTE programs operated under this dual enrollment system, including the HVAC program that 
granted students both a high school diploma and a college certificate at the same time. 

Articulation Agreements 

Unlike dual enrollment, the articulation agreements between West District and WCC spanned 
nearly every high school and every CTE program offered by WCC. This state considered 
articulation agreements to be an integral component of POS, and thus required that all eligible 
programs be articulated. According to one state official, if a high school CTE program did not 
meet standards for articulation, “It’s incumbent on the high school to get that program up to 
standards.” 

Every spring, WCC collected high school student applications for credit from articulated classes, 
provided students with a WCC student identification number, and added each student to the 
relevant college course roster. Over the summer, student grades were sent to the college and 
entered into the system by WCC personnel, even for those students who did not meet the credit- 
earning threshold of a B. WCC then sent official college transcripts to all students who earned 
above a B. Throughout this process, all students whose applications were sent to WCC were 
assigned a WCC student ID number that remained in the system for five years, even if their 
grades did not qualify them to earn the college credit. 
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Articulated courses had to be reviewed at least every three years. WCC worked with the West 
District CTE office, school-level coordinators, and faculty on these reviews. Although most 
agreements were district-wide, others were school-specific because the resources or facilities to 
offer a given course or program were only available at one or two high schools. 

Amaranth’s principal described the close relationship he and his faculty enjoyed with WCC with 
respect to program articulation: “We work really closely with WCC. We meet with them 
quarterly. They come out here. We have a full CTE meeting with their liaisons, getting 
applications, setting goals. Last year I think we had 28 kids [apply for credits], but right now I 
have 160 applications for this year because we understood what we were doing a little bit better.” 

A WCC department chair offered his perspective on this close working relationship: 

Our position has always been a solid connection with our feeder units, which are the high 
schools... Look, some high schools want to go their own way and teach their own stuff 
and are more fine arts oriented, if you will, and God bless them if that’s the direction they 
want to go in. That’s fine. But if they want to embrace a more commercial, career- 
oriented kind of modality, then we’re there for them and we help them any way we can. 

The dean of occupational education at WCC said that he and his faculty designed courses to meet 
industry demands, but could be flexible about methodology when it came to articulating with the 
district: “I know some colleges and universities will not articulate unless the timeframe is the 
same or the textbooks are the same. That’s not our approach. We look predominantly at the 
outcomes. We don’t mandate textbooks. And frankly, if you get those outcomes in a semester, 
great. If it takes a year, two years, fine.” 

Although college credits were automatically transcripted at no charge to students, students still 
had to apply to receive the credits they earned. According to reports by the WCC articulation 
coordinator, the majority of credits accrued by West District students were granted. However, we 
heard from both intervention and comparison teachers and counselors that students sometimes 
lacked the motivation to fill out the paperwork to receive the credits. These participants felt that 
some students did not see the value of the credits, not only financially, but also because these 
students planned to attend out-of-state, top-tier universities, where these community college 
credits would not transfer. 

Lead to an Industry-Recognized Credential or Certificate at the 
Postsecondary Level, or an Associate or Baccalaureate Degree 

West District’s goal was for each program of study it offered to lead to a postsecondary degree 
and opportunities to earn both dual credit and industry-recognized credentials. Although the 
district, WCC, and business and industry partners strongly supported industry-recognized 
credentials, one retired district administrator who was instrumental in establishing the district’s 
plan for POS believed that they had only made “hit or miss” progress in implementing them. A 
district CTE administrator noted that industry-recognized credentials presented particular 
barriers to implementation, including high costs, which he believed the community had to 
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embrace, as well as challenging curricula that might not fit a standard school day. During the 
time of this study, the district did not track credential acquisition, but administrators indicated 
that this was a future goal. 

As a supplement to industry-recognized credentials, the state also provided CTE competency 
certificates to POS completers at the conclusion of a terminal course; teachers filled these forms 
out, attesting to students’ acquired skills and quality of work. Some study participants questioned 
whether these certificates truly signaled students’ career readiness to employers. 

Industry Involvement 

West’s advisory boards actively supported the implementation of industry-recognized 
credentials. During a site visit in the second year of our study (see Castellano et ah, 2011), we 
observed a meeting of the district’s construction technology advisory board in which members 
discussed the feasibility of offering an energy auditing certification in response to a recently 
passed state energy conservation law. Committee members also debated how they might engage 
OSHA-certified seniors in conducting safety inspections at work sites. Forklift certifications and 
NCCER national certification were also discussed. As Sky’s construction technology teacher, an 
advisory board member, commented, with these and other credentials in hand — CPR, OSHA, 
NCCER, forklift operator — his students could graduate and immediately qualify for first- or 
second-year union membership and well-paying jobs. Construction technology advisory board 
business partners interviewed in the third year of the study affirmed the high value of industry- 
approved technical assessments and credentials, making particular mention of the success of the 
district’s vertically aligned HVAC program at Fuchsia, the comprehensive high school located 
next door to a WCC satellite campus. One of these partners also argued for a new certification in 
energy conservation (e.g., NACEP; National American Board of Certified Energy Practitioners). 

The Local Community College 

WCC was a strong supporter of the district’s implementation of dual credit and industry- 
recognized credentials as a means of promoting vertical alignment between high school 
programs and its own advanced programs, and saw this partnership as a means of providing 
“stackable credentials” leading to a wide variety of degree options and occupational exit points. 
As one dean stated, the college believed that industry-approved assessments and credentials — for 
example, NATEF’s ASE-preparatory curriculum — were vitally important to students’ workforce 
preparation. Yet not all of the college’s occupational programs — including those aligned with 
district programs — were aligned with industry certifications, partly because some industries had 
no national-level certifying body, such as WCC’s photography program. Both the dean and 
photography department chair noted that a college diploma had no signaling power in that 
industry, whereas a professional portfolio, which did, did not translate into Perkins 
accountability measures. 

Available Credentials 

In addition to those already described, West provided a wide variety of industry-recognized 
credentials in many program areas. Depending on the school (POS or comprehensive), West’s 
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automotive programs were based on NATEF’s ASE- and GST-preparatory curricula; ICAR 
(Inter- Industry Conference on Auto Collision Repair) was also cited at Sky. All district high 
schools with culinary programs, including the POS high schools, offered the National Restaurant 
Association Educational Foundation’s ProStart curriculum and ServSafe certification. Navy’s 
early childhood education program offered both college credit at WCC and the internship hours 
needed — provided students acquired their health card and sheriffs card — to sit for an early 
childhood license at age 18. Navy’s hospitality program was certified by AHLA’s Educational 
Institute. Azure’s legal studies program qualified students to sit for their paralegal license at 18. 
In the district’s IT programs, the available credentials most often mentioned were C++, CISCO, 
CompTIA A+, Microsoft, Novell, and Oracle. Health sciences programs offered preparation for 
the CNA and CPR certification. Students in Navy’s medical program could obtain a medical 
assisting certificate, but its biotechnology program had no industry credential; instead, 
biotechnology students could earn AP science credits. 

Ruby, a comprehensive school, housed several CTE magnets that allowed students to earn both 
dual credits and certifications. These included an aviation program which offered students the 
ground school training needed to sit for an FAA certificate (often referred to as a private pilot’s 
license); a PLTW-certified pre-engineering program; and an Emergency Medical Technician 
(EMT) program in which students prepared for CPR and EMT-Basic certificates. Ruby was 
investigating the possibility of offering a certified aviation maintenance program as well as 
certification for its sports medicine track. 

Student Perspective 

Students interviewed in 10th grade were mostly ignorant regarding the certificates or credentials 
they could earn in their programs; at this point, these students were still taking core courses. By 
senior year, students appeared more knowledgeable about industry-recognized credentials. At 
Sky, one computer networking student commented on the cost of CISCO certification exams; an 
automotive student noted that his participation in ICAR certification testing cost him nothing, 
unlike the same program at WCC. A Sky health sciences student said that she did not learn about 
the CNA until the end of her sophomore year. As an 18 -year-old senior, she successfully sat her 
CNA exam; by that point, she also had earned college credits that would transfer to WCC or the 
state university and accumulated many hours of clinical hospital experience. She reported that 
she was careful to play up the earning potential of the CNA when helping the school recruit new 
students from area middle schools. 

State and Local Legislation or Administrative Policies 
Promote POS Development and Implementation 


State Policies 

State funding and other resources. This state did not provide legislative funds specifically for 
CTE. Nor did it provide funding for school construction. Instead, West District floated bonds to 
construct new schools, including a number of smaller, POS-focused high schools. Given the size 
of the public expenditure involved, a citizen oversight committee had to approve these 
construction projects before they went up for school board approval. According to a number of 
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interviewees, it was not difficult to convince the committee or the general public of the need for 
the POS high schools. One administrator said that although the district provided a lot of data 
supporting the initiative, they also benefitted from being “at the right time at the right place. . . it 
was good common sense.” At the time the POS high schools were conceived, the region was 
enjoying an economic boom that made gaining public approval easy. Once approved, WCC 
faculty whose programs aligned with the proposed programs helped design the new facilities. 

Although many interviewees praised the idea of POS high schools, some criticisms were voiced. 
A number of interviewees in the schools and at WCC believed that the district had spent too 
much money on facilities they did not need or had created too many programs. Others felt that 
not enough money was available to support all programs. One teacher railed at the low level of 
public education spending in general, bemoaning the state’s position as among the worst- funded 
in the nation. This teacher also felt that private money from industry was lacking. When trying to 
secure high-quality, long-lasting equipment for her students, she had to fight “every step of the 
way, in justification of every piece of that equipment,” as administrators tried to talk her into 
lower quality, lower cost substitutes. Better equipment would last longer because dozens if not 
hundreds of students needed to use it: “You’re training these kids for careers for their life here. 
This is not just, oh, a little elective where they go in and play. This is something these kids want 
to leave and go do for a living, you know? You’re training a workforce here.” 

As the economic recession ground on, schools felt the effects of reduced funding. Some CTE 
programs faced budget cuts or were shuttered. Schools that had not yet opened were reduced in 
scope. Teachers were laid off. Class sizes in all subject areas increased. The planned adoption of 
PLTW in some high schools was delayed. 

Dedicated staff time for POS development; establish formal procedures for the design, 
implementation, and continuous improvement of POS. Several statewide POS planning and 
implementation workshops were held in an effort to bring the comprehensive high schools on 
board with POS. Four people per school attended these workshops: a curriculum administrator, a 
counselor, a CTE teacher, and a department chair. Personnel from the community colleges and 
the state university system also participated, but industry partners did not. After the workshops, 
each school chose a contact person who was tasked with overseeing POS implementation; these 
staff members received time and resource support. The state’s largest challenge was POS 
implementation in its more rural districts, where highly skilled teachers, modem equipment, and 
business partnerships were less available than in its larger cities and counties. One POS high 
school principal suggested that regional skills centers were an option for such areas. 

Provide resources for long-term sustainability of POS. Participants noted that a major challenge 
to implementing POS was learning the rules and regulations of the different federal and state 
funding sources that supported them, then learning how to do without those funds when they ran 
out, either by design or due to the recession. 

Although there was no set-aside in the state’s education budget for CTE, the district CTE 
director said that the state supported CTE generously and sought progress and accountability 
reports on POS implementation. In response, West encouraged POS implementation in its 
comprehensive high schools. When schools requested funds for new equipment, the CTE 
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director weighed such purchases against other expenditures that could increase student 
engagement, sometimes suggesting alternatives such as supporting a CTSO that helped students 
gain leadership skills. 

Other relevant state policies. As in other western states, districts and other local education 
agencies in this state tended to exert strong local control. The state CTE director believed that a 
little more top-down authority might help put “more structure in place.” During our study, the 
state was indeed seeking to create a more coherent system by standardizing CTE courses, course 
names, and sequences. The state also defined a CTE concentrator as a student who completed 
two credits, one of which was in an exit-level course. West District adopted this definition of 
CTE concentrators in its comprehensive high schools, but for the POS high schools, many 
programs spanned the entire four years and earned students four credits; the remaining programs 
were three-credit programs. 

District Policies 

Local funding and other resources. As the district’s POS high schools and magnet programs 
began coming online, they received extra support in the form of temporary seed money. By 
chance, this seed money became available just as regular school budgets were being slashed due 
to the growing recession. Sky, the POS high school that served as a model for others in the 
district, was not slated to receive this extra support because it was not a new school. However, 
the aging Sky had serious physical plant issues, and administrators there convinced the board of 
education to support the school at the same level as the new POS high schools. Overall, this extra 
funding shielded the POS high schools from some of the worst effects of the recession. With that 
said, some programs at the POS high schools faced challenges securing equipment and supplies. 
One teacher in a new hospitality POS said it took a couple of years for the district to approve 
certain purchases he wanted to make because it was not used to requests for items like pipe and 
drape for separating booths at conventions. 

Establish formal procedures for the design, implementation, and continuous improvement of 
POS. West’s CTE director introduced the district’s plan to expand POS during visits to all of its 
high schools. He and his team reviewed each school’s CTE offerings with the curriculum 
administrator, the head counselor, and the CTE chair, going over POS requirements to detennine 
if what they were currently offering could become a POS or if some singleton courses could be 
eliminated. In some cases, the CTE director would say, “We think that this is not a program of 
study. We recommend you go in this direction.” Each school then held staff meetings to share 
this information. During these meetings, staff members reviewed four- year POS course 
sequences of CTE and core academic courses and their respective postsecondary articulations. 
The CTE director said this process was the district’s way of encouraging college readiness and 
preparation in the CTE programs at the comprehensive high schools. 

Bringing POS to all of the comprehensive high schools involved significant, continuous 
curricular upheaval: Courses were revamped and/or renamed, and program sequences came and 
went. The move to POS became more difficult when the district was forced to cut block 
scheduling from its budget, which only the POS high schools were able to maintain. Without 
block scheduling, the comprehensive high schools found it difficult to offer programs at the same 
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level of intensity. However, over the course of our study, CTE programs at comprehensive 
schools grew in their capacity to offer POS. Once less well funded than the POS high schools, 
they benefited as the district sought to implement more components of POS in comprehensive 
high school settings. 

Ensure opportunities for any secondary student to participate in a POS. West designated one 
POS high school to serve each of its several regions. During the course of our study, regional 
boundaries were in flux, which meant changing school catchment areas and subsequent shifts in 
school populations. Although our cohort remained grandfathered at their school of choice, one 
POS high school counselor lamented that new enrollments were shrinking because students in 
the school’s new catchment area were happy with their zoned high schools. Although the POS 
high schools were open to students from all over the district, students had to provide their own 
transportation. As such, the POS high schools competed for enrollments not only with the 
comprehensives, but also with each other. 

Recruiting efforts by the POS high schools and CTE magnets were both shared and individual. 
The district sponsored school fairs at malls across the region. Recruitment counselors from the 
POS high schools also visited the middle schools together to let students know about their 
programs. Individually, each POS high school held open houses for prospective students and 
parents. Such events were sometimes hosted by the school’s culinary and hospitality programs, 
as at Navy, but all POS were showcased. Participants at one school noted that their CTSOs often 
took part in local parades, which gave the school additional exposure in the community. 

Organizationally, the POS high schools fell under the authority of both the district’s magnet 
office and its CTE office. The fonner set recruiting schedules and eligibility requirements and 
ran the district’s placement lottery. The CTE office provided curricula, tools and equipment, and 
funds. Although the POS high schools sometimes perceived a lack of a shared, cohesive 
philosophy across the two offices, most of the POS high school principals reported receiving a 
great deal of overall district support for CTE. 

As we have described elsewhere in this and previous published reports (e.g., Castellano et al., 
2011), the district’s lottery admissions process was not entirely random. Eligibility requirements, 
although not onerous, disqualified some students. West used a two-tiered point system for 
detennining eligibility for what it deemed “standard” or “rigorous” POS. Both types of 
applications assigned points for attendance, behavior, GPA, and test score results, but eligibility 
requirements for rigorous POS also included eighth-grade core subject grades. For example, 
biotechnology and engineering POS had eligibility requirements in math or science because of 
the work students would be expected to do in those programs. Standard POS had no core subject 
grade requirements. Eligibility requirements effectively increased the proportion of high- 
achieving students at the POS high schools, which caused resentment among some 
comprehensive high school principals and prompted accusations of creaming. 

Require secondary students to develop an Individual Graduation or Career Plan. The district 
required all students to develop a four-year graduation plan in the ninth grade. These plans were 
completed online and ideally reviewed annually for changes. We found annual reviews occurred 
more often for the POS students we spoke with than for comparison students. 
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Some policies did not lend themselves to long-term college and career planning. At the middle 
school level, in particular, restrictions on the use of postsecondary Perkins funding prevented 
WCC faculty and staff from sharing information about POS options with middle school students. 
Because students applied for entry into POS in the eighth grade, this was an opportunity lost. 

Provide resources for the long-term sustainability of POS. Funded by public construction 
bonds, the district’s POS high schools were purpose-built to foster POS and curriculum 
integration by co-locating CTE and core academic teachers in the same area of each building. 
This closeness fostered teacher collaboration and student cohorting. Elected officials invited to 
tour these facilities became lifelong supporters when they saw what they made possible, 
according to a district administrator who had been instrumental in developing them. 

Other relevant district policies. Some of the POS high schools implemented policies that put 
failing students on academic probation. Students who could not achieve a passing grade or 
overall 2.0 GPA in the subsequent semester were sent back to their zoned high schools. 

In the POS high schools, students began their programs in either ninth or 10th grade. In either 
case, few opportunities existed to transfer into a different program. For those who began their 
POS in ninth grade, switching programs meant taking the introductory (ninth-grade) class at the 
same time as the mid-level (lOth-grade) class in order to catch up. For those POS that began in 
the 10th grade, ninth-grade students often took introductory courses in several program areas as a 
means of sampling their available options, choosing their POS at the end of ninth grade. In these 
cases, switching programs later was not allowed. 

The district’s internet policies — specifically its adoption of firewalls, content restrictions, and 
student privacy-related rules — were cited as a barrier to some programs, including those in 
broadcasting and journalism. A media teacher reported that her program lost out on opportunities 
to showcase student work because students were blocked from uploading films to contests. 

Ongoing Relationships Among Education, Business, and Other Community Stakeholders Are 
Central to POS Design, Implementation, and Maintenance 

Business and Industry Partnerships 

Partnerships with business and industry were vital to the functioning of West’s CTE programs 
and especially to the development and implementation of its POS high schools. The district’s 
advisory boards, comprised of business and industry, postsecondary, secondary, and community 
partners — were reconstituted prior to the time of our study from some 30 committees to about a 
dozen. These committees generally met monthly to discuss new and emerging program-related 
initiatives and to review curricula, standards, articulation agreements, work-based learning 
opportunities, industry-recognized credentials, and other program-related topics. In every 
interview in which they were discussed, advisory board members were recognized and valued for 
their professional and technical expertise and the assets (including material and financial 
donations) they brought to the district’s POS and CTE programs. Advisory board members 
helped establish the themes around which each of the district’s POS high schools were organized 
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and supported the purposeful design and construction of the buildings themselves. Although the 
district was ultimately responsible for both curricular and funding decisions related to its 
programs, as one district CTE administrator noted, the advisory boards used their clout to lobby 
for and preserve programs that impacted their future business. 

A district CTE administrator saw the district’s CTE academies as creating public-private 
partnerships that met educational and economic goals and benefitted students, industry, and the 
larger community. One such partnership was discussed at the construction technology advisory 
board meeting that we attended in the second year of the study. At this meeting, an industry 
partner with a solar energy business presented a proposal to build a solar photovoltaic system in 
a modular classroom that would be fdled with energy-saving devices donated by industry. The 
classroom would then serve as a facility for private sector training outside of the school day. 
Advisory board members considered whether the proposed project could be expanded to 
additional modular classrooms, each powered by an alternative energy system (e.g., geothennal, 
hydro, wind) that would drive an element of the curriculum. 

Overall, industry partners provided not just technical expertise and curricular input but also 
funds, equipment, work-based learning, advising and mentoring, and jobs for graduates. We 
noted evidence of this generosity at every school visited, both at the POS high schools and in the 
regular CTE programs or magnet POS in the comprehensive high schools. However, schools 
differed in the intensity of their partnerships with industry and the means by which they 
maintained them. 

Business partnerships at POS schools. Sky and Navy employed structures and/or dedicated 
personnel to make and maintain relationships with businesses, the community, and 
postsecondary partners. Most notably, Sky’s career center, developed by its current director and 
partly constructed and networked by Sky students, served as a model for other career centers in 
the district. Calling it “the heartbeat” of the school, the director described how business partners, 
many of them advisory board members, volunteered their time and expertise for mock interviews 
that Sky offered to all students. She numbered some 45 business and industry personnel that she 
could count on to give critiques and career advice to students; some even offered jobs to students 
who impressed them. For the school’s automotive program, for example, the career center 
regularly hosted an interview event for juniors interested in internships at area car dealerships 
and shops; service managers came and conducted panel interviews in which they ranked 
candidates on their professionalism, resumes, and technical skills. Even during the recession, the 
school’s partners continued to post positions on the school’s jobs board, offer funds to pay for 
student scholarships, and volunteer their time: “Anything we ask of them, they’re there and assist 
us... we don’t have to beg.” The lead instructor of the school’s commercial construction 
program, a member of the construction technology advisory board, described how local 
businesses and trade associations held the program’s graduates in high esteem and were anxious 
to support the program with curricular input and generous donations of top-of-the-line 
equipment. This teacher, like many CTE teachers at the POS high schools, came from a business 
and industry background and maintained strong ties with area construction firms, architects, and 
unions. Teachers like him drew on their rich work experiences to connect students with other 
professionals with knowledge and employment opportunities to offer. 
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Business partnerships at comparison schools. Relationships with industry partners were no less 
rewarding at the comprehensive high schools, but the mechanisms by which they were created 
and supported were sometimes less fonnal than at the POS high schools. A notable exception 
was Ruby, where teachers in the aviation academy, all retired military pilots, worked with local 
postsecondary institutions, flight schools, and aviation firms (one of which was owned and run 
by a Ruby graduate) to offer their students greater rigor and opportunities for hands-on learning. 
Ruby’s medical academy was staffed by teachers who maintained active ties to the EMT, 
firefighting, and sports medicine communities. In another noteworthy case, Fuchsia’s principal 
was in the process of converting her school’s CTE programs into POS-like academies with the 
help of WCC and area organizations and businesses. One of these businesses, a major apparel 
retailer known for its positive, customer- focused corporate culture, worked with the school to 
develop and implement a school-wide “core values” campaign. This school was also home to 
both a career center that coordinated internships and job shadowing opportunities and an 
entrepreneurial education program based upon the concept of structuring schoolwork like an 
office environment. This program counted many business and community partners as active 
supporters, including banks, the local government, and a Toastmasters club. At other 
comprehensive high schools, administrators and teachers spoke wannly of the support of area 
auto dealers, hospitals, restaurants, and hotels, but relationships with these partners were largely 
managed by individual teachers. An exception was Vennilion’s advisory board, managed by its 
principal, that included representatives from the area’s tourism and hospitality industry. 

Community Partnerships 

Our observations suggested that West’s high schools, particularly its POS high schools, were 
held in high esteem by the community. Sky, in particular, the district’s original POS high school, 
had a long-standing reputation for academic and technical excellence and student achievement. 
Sky’s success and popularity prompted the district to replicate its success with Azure, the area’s 
first technology-focused POS high school, and later with the POS high schools. Community 
support was part of this process. As one retired district administrator recounted, when the district 
first conceived of these new schools, it held public meetings at which it solicited input; it also 
drew on its advisory boards for support. Having been built to support regional economic and 
workforce development needs, the POS high schools were expected to gain additional public 
support over time. As a retired administrator said, “I really think that we’re going to continue to 
see such tremendous student success, and because we’ve had a really broad involvement of the 
community — both private and public — I think people would be up in arms if we said we weren’t 
going to continue to do what we’re doing.” 

All of West’s high schools appeared to offer valuable resources to their communities. At Ruby, 
the principal related how the school provided infonnational and enrichment activities to parents 
and the community (e.g., language programs, computer training, gang prevention forums). Navy 
employed a full-time business and community partnership coordinator who, in addition to 
seeking and maintaining partnerships, coordinated special community-wide outreach events that 
highlighted the school’s programs. Navy’s culinary and hospitality programs were especially 
active in catering and hosting such events, like a recent banquet honoring Holocaust survivors 
that drew on students’ work in their history classes. One of the culinary program’s chef- 
instructors commented that in the past year, students had catered at least 10,000 covers by 
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hosting this and similar events. Such interactions with the community were perceived as valuable 
ways to teach students how to interact socially with the public and potential future employers. 

Postsecondary Partnerships 

WCC and the local four-year university were no less instrumental to the successful development 
and implementation of West’s POS high schools. A retired district administrator explained how 
in the early days of developing POS, the district worked hand-in-hand with the college and the 
university to ensure that its programs aligned with postsecondary programs not only for the 
purpose of generating dual credit, but also to meet institutional needs for well-prepared 
applicants. She told a story in which the dean of engineering at the four-year university, from 
whom the district had sought input on appropriate courses for students in a construction 
engineering pathway, acknowledged the value of the skills the POS would offer: 

We went to the dean. . . and said, “What do we want to make sure we offer our students?” 
And he said, “Well, you absolutely have to have the rigor, and make sure the kids have 
access to AP Calculus.” And. . . he went through the courses that he felt were really 
important. He said, “But you know, it’s going to make such a difference to have students 
who actually will understand and have had experience putting things together. . . it’s not 
just going to be a mathematical formula for them. . . They’re actually going to have a 
much better sense that things are going to be able to work together by understanding how 
things are built.” He said, “I wish all of my students had that opportunity.” He said it was 
going to make them far better engineers. 

Both the university and WCC supported the district’s initiatives because they created seamless 
pathways to their programs; for WCC in particular, credit transfer opportunities helped boost 
FTEs and were deemed a sound investment in future enrollments. As such, WCC took many 
steps to simplify the process of earning dual credit, including waiving the low fee associated with 
obtaining the credits and automatically transcripting them while students were still in high 
school. The college also employed a full-time coordinator who kept principals and teachers 
informed, distributed brochures, and collected paperwork from each of West’s high schools. 

Over time, WCC’s articulated offerings evolved in response to concerns regarding student 
preparedness and curriculum alignment at some schools. As the articulation coordinator 
explained, WCC recently signed a new joint policy agreement with the district that would require 
students to take a full sequence of courses — not random or unconnected classes — in order to 
qualify for credit. Students could apply as juniors or seniors to receive that credit. Although this 
change would not affect students in our cohort, it was made to satisfy concerns that high school 
freshmen and sophomores were not likely to retain enough of what they had learned to justify 
earning the credit. During our study, the college ended the practice of creating district-wide 
programmatic articulation agreements. As the dean of the college’s engineering technology 
division explained, the college was now quite selective about the articulation agreements it 
struck with specific high schools, requiring them to be taught by college-trained high school 
teachers and to follow college-approved curricula. In return, district high school students who 
enrolled in these aligned programs and applied for the college credit could graduate with “a leg 
up” on a college degree — in some cases, enough credits to bypass all or most of the introductory 
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courses required in a given college program. 

College administrators and faculty actively participated on district advisory boards and 
contributed greatly to the development and revision of state and district CTE curricula. WCC’s 
culinary department chair, for example, described how he worked with both the state and the 
district to revise curricula, course titles, assessments, and standards for baking and pastry 
programs. He also described how WCC worked closely with high school culinary programs in 
order to ensure that the students coming to the college with advanced standing (e.g., articulated 
credits) would arrive with the preparation and skills needed to succeed in a professional, 
restaurant-focused course of studies. Although he rated the ongoing curriculum revision process 
as “fair to good,” he voiced concerns regarding inconsistencies across high school culinary 
programs, especially those led by FACS teachers with no background in the restaurant industry. 
At WCC, by comparison, all culinary instructors were required to have professional experience. 

Postsecondary partnerships at POS schools. At our intervention sites, many of the teachers we 
interviewed mentioned participating regularly in district- and state-level curriculum development 
and revision workgroups with postsecondary colleagues, usually from WCC. A number were 
qualified to teach community college classes. Many teachers also mentioned enjoying warm 
relationships with postsecondary colleagues to whom they could turn for student mentoring, 
assistance with projects, and college advising. Nearly all of the POS high school programs 
allowed students to earn credits at WCC. One that did not was Navy’s biotechnology program, 
which was developed with input from the local four-year university. For this program, AP 
courses were the only college credits available; the program’s director explained that both WCC 
and the university preferred that students take postsecondary anatomy courses. In another 
exception, the director of Navy’s early childhood education program tried unsuccessfully to 
articulate her program with WCC. Instead, she designed its curriculum according to the pattern 
of her own undergraduate studies; when interviewed, she was in the process of negotiating with a 
state university to accept some courses for credit. Across all POS high schools, teachers were 
well infonned regarding credit transfer opportunities; interviewed students also seemed well 
informed, especially as upperclassmen, when such opportunities were more likely to be offered. 
Infonnation was available in lower grades, however: During our ninth-grade site visit, we saw an 
Azure computer applications teacher present a detailed lesson on the value of dual credit. This 
teacher, who had just attended an informational workshop offered by WCC, directed her class to 
the college’s website, where she showed them which of their classes aligned with those offered 
by the college and which could be transferred to in-state universities. She then described how 
nearly 70 Azure students had taken advantage of the program during the previous year, at a 
savings of $19,000. Her lesson also included an exploration of how much students could expect 
to earn after acquiring a degree in some of WCC’s career programs. 

Postsecondary partnerships at comparison schools. As with business partnerships, connections 
with postsecondary appeared somewhat stronger and more fonnalized at Ruby, with its magnet 
POS, than at other comprehensive high schools, although many teachers at the comparison sites 
mentioned having colleagues at WCC or participating in professional development and extended 
learning there. All schools were fully informed about the college credits available through their 
programs. With the exception of Ruby, where the magnet coordinator worked with POS teachers 
to align their programs with WCC and other local postsecondary institutions, we interviewed no 
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comprehensive high school CTE teachers who reported collaborating directly with their 
postsecondary counterparts or serving as members of any district advisory boards. 


Sustained, Intensive, and Focused Professional Development Opportunities for 
Administrators, Teachers, and Faculty Foster 
POS Design, Implementation, and Maintenance 

Professional Development on POS 

Unique among our three study districts, West was the only district in which POS were the focus 
of a state-level professional development workshop in which district staff, school administrators, 
guidance counselors, and CTE teachers participated. As noted by the state’s CTE director, the 
state and district worked closely together to coordinate the event, to which four-person teams of 
school staff came from nearly every high school. Invited schools included regular comprehensive 
high schools, comprehensives with magnet POS, and the POS high schools, the leading edge of 
the state’s POS initiative. At the meeting, state standards, processes, and timelines for POS 
implementation were laid out; as the state CTE director reported, the goal was to reach “a 
common understanding” and determine “next steps. . . because this is a process, and right now we 
have decided that the best thing we can do is to be on call.” Post-meeting, the state’s primary 
goal was to provide districts with as-needed technical assistance, host twice-annual meetings 
with secondary and postsecondary CTE administrators, and maintain an accountability system 
that kept the initiative moving forward. 

State and district administrators believed that schools participating in this meeting recognized the 
value of POS, as one district CTE administrator stated: 

I was in shock at how quickly they accepted it. . . the counselors and the administrators, I 
think they just get blinded by “I’ve got to build a master schedule,” and “I’ve got to do 
this and avoid conflicts,” and whatever. And once they looked at it, they saw it as an 
easier way to market programs to students and parents; they saw the connectivity to 
postsecondary opportunities, and I think they saw how much easier it will be to build a 
master schedule. . .without having a bunch of unique, orphan courses out there. 

Following this meeting, the district sought to deliver school-specific professional development to 
ensure that principals understood the integrity of the state’s POS and would not only maintain 
and update their curricula, but also preserve their alignment with postsecondary and industry 
certifications. To accomplish this, district CTE administrators first met directly with principals, 
then dispatched district CTE coordinators into the comprehensive high schools to work directly 
with the assistant principals for curriculum, lead guidance counselors, and CTE chairpersons to 
identify which of their programs, courses, and industry certifications (where available) fit within 
the state’s POS framework. According to another district CTE administrator, the district strove to 
offer schools a “menu” of POS choices that would allow those schools with different daily 
schedules, smaller CTE programs, or programs with limited industry certification options the 
flexibility to structure their POS as two-year versus three- or four- year POS sequences. A related 
goal was to identify and cut resource-draining “orphans” — site-specific, singleton courses. 
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The Local Community College 

The district was soundly supported in its efforts to promote awareness of POS by WCC, which 
employed a dedicated articulation coordinator who visited each of the district’s high schools to 
disseminate infonnation to principals, guidance counselors, CTE department chairs, and teachers 
about how the high schools’ programs aligned with WCC programs. This coordinator also 
collected completed applications from students applying for college credit. Her role also 
included monitoring whether schools were properly sequencing their courses and advising 
students regarding the appropriate level of preparation they needed to qualify for college credit. 

A WCC department chair stressed that the college’s participation on advisory boards and its 
active role in training district teachers and certifying their courses ensured that students would 
arrive well-prepared to succeed and with enough credits earned during high school to allow them 
to start taking advanced classes. 

General District CTE Professional Development 

West scheduled four full-day trainings during the year for all school personnel: two school- 
focused, and two curriculum-focused (in the case of CTE, professional development was 
conducted by program area; e.g., automotive, health sciences, culinary). Other scheduled training 
opportunities included regular department meetings, monthly advisory board meetings with 
postsecondary and business and industry partners, school-based career center trainings (at those 
schools with career centers), intensive summer sessions, academy-specific trainings (e.g., PLTW, 
NAF), and annual statewide conferences (Navy culinary and hospitality students hosted and 
catered one of these events, which attracted over 500 attendees). All alternatively certified 
teachers received expert teacher mentoring and intensive professional development support in 
instructional strategies and classroom management techniques. The district also supported 
teachers in their pursuit of professional development at local, regional, and national 
postsecondary institutions, and from nationally recognized industry certification programs (e.g., 
Adobe, Apple, Microsoft, NATEF). WCC was most often mentioned as a trusted source of 
professional development. Some teachers noted that they had pursued and paid for continuing 
education on a variety of topics on their own. 

Major topics of professional development mentioned by study participants included project- 
based learning (most intensely implemented at Navy), curriculum integration, mathematics, 
ESOL, the use of student assessment data, instructional technology, credit transfer opportunities, 
and industry certifications. 

Project-Based Learning 

As we described in Castellano et al. (2011), Navy’s POS — and the school campus itself — were 
built around the concept of contextualized project -based learning in which academic curricula 
were integrated into CTE program areas. Several times a year, each year, Navy’s various small 
learning communities came together to put on school- wide project-based learning-focused 
events. Navy’s principal said that he hired teachers (especially talented candidates from business 
and industry) selectively for the right attitudes toward project-based learning and then sought to 
cultivate their practice of the approach through intensive, continuous professional development, 
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including a summer institute, regular staff development days, and support from the school’s 
dedicated project-based learning coordinator and staff trainers. Teachers said that they 
participated in both formal and infonnal teacher-to-teacher mentoring and cross-unit 
collaboration. Although district CTE personnel noted that project-based learning was a key focus 
of professional development in the other POS high schools and at the comprehensive high 
schools, few participants outside of Navy mentioned participating in such training. Ruby’s 
magnet coordinator said that she had participated in online-only professional development on the 
approach and was trying with some difficulty to help teachers in her school’s programs 
implement it in their classes. 

Professional Development for Guidance Counselors 

Like West District teachers, guidance counselors participated in a minimum of four full-day staff 
development sessions each year, plus supplemental professional development on a variety of 
topics and opportunities to earn college credit that could be applied to keeping their counseling 
licenses current. One Navy counselor interviewed in April mentioned having participated in 
some 1 8 training events so far that school year. Professional development topics most often 
mentioned included scheduling, testing, transcript reviews, credit retrieval, college preparation 
(planning, testing, advising, financial aid, scholarships), school safety, mental health counseling 
issues, and career advising. Counselors also described participating in trainings or infonnation 
sessions addressing CTE-related topics like credit transfer opportunities, internships or 
apprenticeships, postsecondary occupational programs, and local work opportunities. 

Systems and Strategies to Gather Quantitative and Qualitative Data on Both POS Components 
and Student Outcomes Are Crucial for Ongoing Efforts to Develop and Implement POS 

State Assessments 

In this state, 10th graders took state proficiency exams in reading, math, and science. Those who 
passed were allowed to graduate, but those who did not were retested. If students did not pass 
any one of the required exams by the end of twelfth grade, they received a certificate of 
completion instead of a diploma. 

Academic assessments. In addition to the state lOth-grade assessments, West District also 
administered interim assessments and common semester assessments for core academic classes, 
but not for CTE or POS classes. The district began using interim assessments during our study in 
order to provide teachers with data on their current students. Results were provided instantly at 
the student and question level. Teachers then sat together as departments and analyzed their 
results to identify the areas in which their students needed additional help. Those teachers with 
consistently high-scoring students were observed and their strategies adopted by other teachers. 
The goal of this process was to address student needs early enough in the semester to make a 
difference in student achievement. 

The district also instituted common semester exams for core courses across all schools in order 
to help students who changed schools stay on track. District personnel said that these exams were 
a teaching tool, not a high-stakes exam. Students did not perfonn well on the first administration 
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of the tests, so the district responded by providing schools with more learning tools to support 
them, including software that allowed students to re-take courses such Algebra I and Geometry 
in an online fonnat that offered a personalized curriculum. 

Although academic teachers spoke of using authentic assessments like plays, drawings, and 
Socratic discussions in their classes, most students we talked to said that core course assessments 
were almost exclusively paper and pencil tests made up of multiple choice or essay questions. 

Content Standards Define What Students Are Expected to Know and Be Able to Do to 
Enter and Advance in College and/or Their Careers 

In West District, college and career readiness was most often described as preparing students to 
graduate on time armed with the academic and technical knowledge and workplace readiness 
skills needed to pursue higher education without the need for remediation and to successfully 
enter the job market. Many participants indicated that they wanted their students to graduate 
fully aware and prepared to take advantage of a broad range of potential educational and career 
options. Indeed, in speaking of the key transition point from high school graduation into the next 
level of education, the state’s director of CTE noted that they were “going back to defining — 
organizing, defining, marketing those exit points we’re talking about in career and technical 
education a little better — about having these students prepared, college ready, for that first exit 
point. The first of. . . hopefully it will be many, but it may be the first before the workplace.” 

Overall, three elements were frequently mentioned as supporting college and career readiness: 

(a) work-based learning opportunities (facilitated by business partners), (b) the availability of 
credit transfer opportunities and early college experiences, and (c) continuous, well-integrated 
career guidance and counseling delivered by many school personnel. 

Work-Based Learning and Partnerships with Business and Industry 

Although enrollment and persistence in some form of college were stressed as key college and 
career readiness goals, a number of school personnel emphasized the need to prepare students to 
get a job right after graduation, especially given the poor state of the local economy and rising 
college costs. Many interviewees believed that early work-based learning experiences (e.g., 
through internships, clinicals, or school-based enterprises) were key to helping students acquire 
well-paying jobs that could help them pay for college. A district CTE administrator attested to 
the major role that the district’s business partners played in aligning district curricula with 
industry standards and providing opportunities for students to participate in work-based learning 
that could lead to future jobs. Two business partners from the district’s construction technology 
advisory board attested to their commitment to providing students with internships and summer 
programs. 

Dual Credit Earning Opportunities and Exposure to College 

Opportunities to earn college credits or take classes at WCC were also stressed. Dandelion’s 
magnet coordinator described college and career readiness as giving students the opportunity to 
not only thoroughly understand their program area in a way that allowed them to “jump ahead” 
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of other students without such opportunities, but also to earn college credits that would allow 
them to bypass introductory college courses. Another magnet coordinator spoke of how her 
school’s magnet programs prepared students to take “the next step,” whatever their interests 
might be: 

What is it you want to do? We look in terms of whatever we can do that best prepares 
them for when they leave us. You know they talk about being college prepared and we do 
offer the college credit. But again, not every kid’s looking to go to [WCC] . . . Our 
mission of sorts is that kids are well rounded and well prepared when they leave here 
because we do understand there’s a lot beyond the academic that helps to form a student. 

Career Guidance and Counseling 

At a number of West schools, particularly the POS high schools, participants spoke of 
comprehensive guidance and counseling as key to college and career readiness. Although many 
tools and mechanisms (e.g., career interest inventories, a state college and career planning 
website) were described as part of this, a business partner perceived it as a school- wide effort at 
Navy: 


Coming out of this school, I would say I have no concerns [regarding college and career 
readiness]. No concerns to either of those points. . . I would say that in this school, they 
have teachers, they have the administrators, they have the advisory board. [The 
community partnership coordinator] and others get in front of the students a lot. So when 
you talk about counselors. . . they — the teachers, the administrators, the people that come 
in — are counselors for these kids. 

Guidance Counseling and Academic Advisement Help Students to 
Make Informed Decisions About Which POS to Pursue 

As previously described, West participated in a state-led training on POS that included 
principals, assistant principals, and guidance counselors from all district high schools. Over the 
course of the study, the district continued to educate schools about how to work with their 
teachers, industry partners, and postsecondary partners to develop robust POS. As a result of this 
training, many study participants, including counselors, agreed that POS offered a terrific way to 
market CTE to students and parents. Counselors, CTE teachers, and magnet coordinators at the 
POS high schools and magnet POS all took an active role in recruiting and educating students 
about the district’s POS at middle schools and magnet fairs. 

Guidance Counselors ’ Roles and Responsibilities 

Counselors’ caseloads and tasks differed somewhat across the POS high schools. At Navy, 
counselors were assigned to specific program areas and followed the same students for all four 
years. These counselors were “specialists” in their areas; one counselor visited program teachers 
to learn more about their curricula and went on field trips so she could see the connections 
between Navy’s programs and their real-world counterparts. Drawing upon their deep program 
knowledge, Navy’s counselors offered recruiting and training at area middle schools so that both 
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middle school students and their counselors understood Navy’s programs. Counselors also 
reported serving as facilitators of parent-teacher meetings and tracking students’ college and 
scholarship offers. 

Although Azure otherwise broke out student caseloads alphabetically, this school was unique in 
featuring a dedicated ninth-grade counselor who introduced a student-to -student mentoring 
program to draw freshmen into the academically intense culture of the school. The Azure 
counselors we interviewed acknowledged that although they were aware of POS, they mostly 
focused on academic and college-preparatory advising; with the exception of making in-class 
presentations on career planning, they largely left career guidance to teachers. Compared to the 
two other POS high schools in the study, however, Azure had the least well-developed or well- 
informed understanding of POS. This was reflected in criticisms made by one Azure CTE 
teacher who believed that, lacking an adequate understanding of POS sequences, some of the 
school’s counselors and administrators placed students in classes in which they had no interest or 
in higher levels of courses than they were competent to take. 

At Sky, counselors carried alphabetized caseloads and actively worked with teachers to market 
the school’s programs to area middle schools and the larger community. One guidance counselor 
noted that although she did not have the same deep knowledge of the school’s POS as its 
teachers, she considered herself well-informed regarding credit transfer opportunities and 
summer learning options. 

By contrast, at Crimson, a comprehensive high school, the counselor we interviewed seemed 
unfamiliar with her school’s CTE programs. Her work mostly involved scheduling, screening 
potential dual enrollment applications, registering students for classes, and reviewing four-year 
graduation plans. On these plans, students indicated their postsecondary goals (e.g., attending a 
two- or four- year institution; working); she then guided them toward coursework that suited their 
goals. Although she worked with teachers to offer some career guidance, she perceived such 
guidance as more relevant for “business-oriented” (i.e., CTE) students. The school’s one- 
semester, non-mandatory careers class covered resumes, job applications, and interviewing. 

Career Guidance Activities and Personnel 

In addition to the core elements of Perkins-defined POS, West’s programs of study incorporated 
many enriched curricular, pedagogical, and extra-curricular elements that demanded 
considerable time and resources to be implemented well. One of these elements was sustained 
career guidance, development, and exploration, which many POS teachers incorporated into their 
classroom teaching. In some schools, dedicated support staff — like academy coordinators or 
Navy’s business and community partnership coordinator — oversaw partnerships, arranged 
outreach activities, internship placements, and special events, and located resources to support 
programs. Sky infused career guidance into the curriculum through established structures (its 
full-service career center) and required courses (its career-focused 21 st -century skills course). 
Both of these mechanisms served to enculturate students and teach them the behaviors and skills 
they needed to help them advance in college or careers. Schools were also supported by WCC, at 
which a dedicated coordinator worked with principals and contacts within the POS and CTE 
programs — lead teachers or academy coordinators, not counselors — to promote dual enrollment 
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and handle paperwork. 

Classroom-Based Career Guidance 

Classroom observations revealed that teachers used their professional backgrounds to enrich 
learning and skill development. As an example, during a senior medical assisting class at Navy 
in which students practiced taking EKG readings, the teacher — a licensed doctor with a 
background in emergency medicine — offered students feedback that combined technical 
knowledge with frank assessments of patients’ body types and needs (e.g., weight, gender 
differences) and how these might affect treatment. In an interview, this teacher reflected on his 
experiences training medical residents and how Navy’s commitment to professionalizing its 
students influenced his decision to teach there: 

What I’m interested in is creating professionals. . . what I saw. . . when I was working in 
medicine that distressed me was kids [who] got all the way through medical school and 
now they’re residents and they still don’t understand what it is to be a professional. They 
don’t understand their responsibilities. They think they’re supposed to get a coffee break, 
and they’re surprised when I tell them, “You know what? Being a professional is not 
about when you get your coffee break. Being a professional is about giving fiduciary 
responsibility to your patients. You have to treat them first and take care of their issues 
before your own issues, and if you can’t be a fiduciary, you can’t be a professional” . . . 
I’ve always kind of been working against that, so I heard [here at Navy] that “OK, we 
want to create some professionals here and show people how to become professionals”. . . 
that’s what really interests me. 

Many other personnel interviewed at Navy expounded on the school’s culture and curriculum of 
professionalism. The principal believed that Navy students were being given a broad preparation 
that offered them many options and the skills needed to succeed in any career in a rapidly 
changing economy. A media teacher said her program allowed students to do television and film 
work using the same equipment used in industry; the test of their training, she believed, was their 
demonstrated ability to excel in postsecondary and do any kind of related work. 

We also observed expert classroom-based career guidance in the comprehensive high schools, 
albeit mostly in the magnet POS. During an EMT class in the health sciences magnet at Ruby, 
the teacher offered a lively, interactive review of the types of equipment and emergency 
situations encountered by EMTs. Herself an EMT, the teacher discussed the serious ramifications 
of work-related injury — and the importance of safety standards — to any young person seeking a 
career in emergency medicine. Framing her remarks in relation to her students’ educational and 
career plans, she asked them to consider what they would do if they were disabled on the job but 
had no fallback training or education on which to reestablish their careers. 

Other Support Personnel 

Outside the classroom, career guidance and services were offered by personnel other than 
counselors and teachers. Navy and Sky allocated staff — at Navy, a business and community 
partnerships coordinator; at Sky, the director of the school’s career center — who established and 
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maintained school-community partnerships, arranged work-based learning experiences, provided 
career preparatory activities, and oversaw special events. In the comprehensive high schools with 
magnet POS, magnet coordinators took on these roles. Ruby’s magnet coordinator recalled how 
she had helped a health sciences magnet POS student clarify her career interests in pharmacology 
and ROTC, identify specific college programs tailored to her interest, and make sure she was 
taking the right courses to qualify for admission to those programs. She also helped her teachers 
work with WCC to align their curricula. 

Innovative and Creative Instructional Approaches Enable Teachers to 
Integrate Academic and Technical Instruction and Students to 
Apply Academic and Technical Learning in Their POS Coursework 

Interdisciplinary Teaching Teams 

POS high schools. One of the district’s original developers of POS recounted how academic and 
technical teachers served together on the study committee that created the POS high schools. 

This committee decided which program areas would drive the district’s POS and intentionally 
designed the physical structure of the district’s first new POS high school, Navy, in order to co- 
locate academic and POS teachers in a given program in the same part of the building. 

Wearing hard hats, Navy’s future academic department chairs and POS program leaders 
discussed in these meetings what academic and CTE teacher collaboration would look like even 
before the school was fully built. Each of Navy’s two houses contained four POS program areas, 
with math, English, science, and social studies teachers assigned to each house. Co-locating 
academics and CTE made teacher collaboration easier. In interviews, many participants said that 
it was common for academic teachers to assign research projects that students could tailor to 
their POS-related interests. Navy’s principal said that he expected academic teachers to make 
their assignments relevant to their house POS; similarly, he expected POS teachers to ensure that 
academic concepts (e.g., syntax and grammar) were part of their assignments. Students were 
released early once a week to give teachers an opportunity to collaborate, which the principal 
believed was key to integrating curricula and developing cross-curricular school- wide project- 
based learning events. As he noted, Navy’s academic curriculum supported the goals of its POS 
curriculum, not the other way around. By building and defining the school in this way, Navy 
attracted teachers who were drawn to this approach, which in turn helped perpetuate it. 

Other POS high schools did not share Navy’s structure. An English teacher at Azure said that 
their schedule precluded cohorting students around their eight POS. The school’s budget-driven 
move from block scheduling to regular periods meant shorter class times and hence less time to 
work on projects, cross-curricular or not. Other teachers at Azure also mentioned the lack of time 
provided to collaborate across subject areas — teachers there did what their curricula required and 
sometimes had little time or energy for more. 

Sky’s principal noted that prior to his arrival, POS and academic teachers rarely spoke to each 
other. To combat this, he invited POS and academic teachers to open houses in which teachers 
could showcase their work, communicate with each other, and “feel valued.” However, an 
overall lack of regularly scheduled opportunities to collaborate discouraged academic and CTE 
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curricular integration at Sky. An office technology teacher said that she taught students 
percentages and how to read a ruler without help from a math teacher. She wanted to work with 
an English teacher on a publishing project, but saw no opportunity to do so. 

Across the POS high schools, common examples of curriculum integration involved integrating 
math in construction classes, math and physics in robotics classes, and English in law classes. 

We directly observed no examples of team teaching across disciplines at any POS high school. 

Comparison schools. The challenge of integrating academic and technical instruction at the 
comprehensive high schools was summarized by one automotive teacher. He said that with only 
two or three automotive students in any given math class, auto-specific math would alienate the 
rest of the students in the class. Despite this obvious difficulty, some integration of academics 
and CTE did take place: Amaranth’s principal described how his English teachers taught CTE 
and art teachers how to teach writing in their program areas and helped them with strategies to 
get students writing every day. The principal served as a substitute for a physics teacher, 
allowing that teacher a full day to engage in curriculum development with a CAD teacher. On 
some staff development days, Crimson’s academic teachers worked with CTE teachers to 
incorporate academic goals and objectives into their courses. A strong example of curriculum 
integration in the comparison schools was a Fuchsia business class that was integrated with 
English and co-taught by an English teacher — this was the only example of cross-disciplinary 
team teaching we directly observed in West District. 

Student perspective. Students at Navy were aware that teachers worked together to integrate 
academic and technical instruction. One student said that her academic teachers often asked, 
“Who’s in medical?... Who’s in engineering? Because you’ll need to know this.” No student we 
spoke with at Azure mentioned such teacher collaboration. At Sky, students reported that POS 
teachers presented academic material; no students reported experiencing team teaching. 

District perspective. In general, West followed a mandated state curriculum and met state-set 
benchmarks for its courses. Only teachers at Navy appeared to have received intensive training 
in how to meet those benchmarks using project-based learning techniques that blended academic 
and technical content knowledge. Professional learning communities within academic 
departments were being implemented throughout the district during our study. Teachers’ 
responses to this effort were positive, and some principals and teachers saw the value of trying to 
develop such communities for CTE teachers or across disciplines. This had not become policy by 
the time we completed our observations. 

Contextualized Learning Approaches: Work-Based Learning 

Although not a district-wide mandate, work-based learning was integral to many of the CTE 
programs we observed. District administrators noted that they had abundant business and 
community support for work-based learning and were in the process of developing new materials 
to guide these activities, but acknowledged that scheduling remained the biggest obstacle to 
making it a district-wide initiative outside its POS high schools. In most comprehensive high 
schools, non-block schedules made it difficult to schedule any kind of work-based learning, but 
especially off-campus internships or job placements. Staffing was also seen as a challenge, as 
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CTE teachers, aided by release periods, were largely responsible for coordinating work -based 
learning. Dedicated support staff were seen as a solution to the problem, as at Navy and Sky, 
where a business partnerships coordinator and a career center director handled the tasks of 
identifying, vetting, and training business partners, screening and placing students, and 
documenting student outcomes. Navy’s partnerships coordinator believed that perhaps 10% of 
the school’s students participated in internships, whereas far more performed community service 
work for credit (a half-credit for 60 hours of volunteering or a full credit for 100 hours). 

Despite the recession, support for work-based learning among the district’s business partners 
remained high. For one of Navy’s energy industry partners, investing in the district’s CTE 
programs, providing input into district and community college curricula, and committing to 
offering work-based learning just made good business sense because “you don’t want to hire a 
kid and have to retrain them. You want to be able to help get him the tools, and the best way is I 
come along and say, ‘Listen. This is what the industry is doing right now. This is what’s 
happening in energy. You need to be focusing in these areas.’” 

A WCC dean agreed that business and industry made substantial investments in district and 
college CTE programs, each of which had work-based components. In many instances, due to 
liability, practicality, or maturity issues, work-based learning primarily took place at the college 
level. As the dean noted, most CTE programs began in the 1 1th grade and progressed through 
college, with different opportunities for work-based learning, internships, or apprenticeships 
appropriate to each stage. 

Work-based learning varied across schools and programs, but included school-based enterprises 
(e.g., school coffee shops and restaurants, t-shirt and poster design and print shops, student 
stores), on-campus activities (e.g., providing sideline sports medicine assistance at school 
athletic events, performing oil changes on faculty cars in the school’s in-house auto shop, 
shooting and broadcasting school news programs, building and selling furniture), and off-campus 
internships or jobs (e.g., interning for local television and radio stations, hotels, restaurants, 
businesses, law practices, or construction companies; completing clinicals at hospitals, medical 
research labs, elementary schools, or daycare centers). 

Students in Navy’s early childhood and teacher education programs gained experience that 
would allow them to become teachers’ aides after graduating from high school or, drawing upon 
the district’s articulation agreements, pursue additional coursework in preparation for a 
baccalaureate degree and a teaching license. Cost was a factor: All students had to acquire health 
cards and be insured prior to placement. Further, because transportation presented a barrier to 
students pursuing internships at off-site elementary schools, early childhood programs, or non- 
home daycare facilities, many students opted to do their clinical work in Navy’s on-site 
kindergarten. Clinicals involved lesson planning, converting lesson plans into centers, engaging 
in hands-on teaching in the school’s on-site kindergarten, and revising and improving lessons. 
The entire process was designed to culminate in the creation of a year’s worth of curriculum that 
students could carry with them to college. 

According to the schooFs biotechnology instructor and project-based learning coordinator, 
Navy’s biotechnology students were eagerly sought-after for internships — not mandatory in this 
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POS — at regional medical and cancer research centers, where they were able to apply their 
advanced skills to lab work and field research. We met several students in this program who 
were working on research projects with microbiology faculty at the local university. 

Navy’s culinary students were required to complete short internships at major restaurants during 
the second half of their senior year. As an instructor described it, these internships had to entail a 
minimum of 10 hours (2 hours per day for 5 days), but many students chose to work additional 
hours — as many as 4 to 6 a day — because they enjoyed the work. The restaurants’ standards had 
to be stringent. By this instructor’s tally, some 95% of the students who sought a job 
immediately after their internships secured one with their internship employer. Hospitality 
students similarly went out on internship during the second semester of their senior year; these 
students also had to pass an AHLA certification test with an 80% or better. 

Interviewed students unanimously spoke positively of their internships and work-based learning 
experiences, describing both on- and off-campus placements as drawing on the skills they 
learned in the classroom and allowing them to explore career options free of the pressure 
associated with a real job or college major. As one automotive student commented, an internship 
at an area dealership gave him a chance to “see how it is in the real world. Not the high school. 
And actually to see if I’m really interested, you know? If not, I’m just gonna be wasting my 
time.” A health sciences student spoke of how rewarding — personally and professionally — her 
clinical experience in a hospital’s surgery wing was. When a post-operative heart patient suffered 
a panic attack, this student saw what was happening and rushed for assistance: “I got her help as 
fast as I could, and afterwards, the feeling was great, having her say, ‘You were a really great 
help.’ Just knowing that I was part of that. . . And it’s not so much the money, and I didn’t get 
paid for it. I just love the experience and I love seeing their smile. ... I enjoyed being with the 
patients.” 

Contextualized Learning Approaches: Project-Based Learning 

As district CTE personnel and school personnel at all sites informed us, project-based learning 
was the focus of a district-wide initiative that began with the POS high schools and was 
gradually being rolled out to CTE programs and magnet POS at the comprehensive high schools 
through district-provided professional development and CTE mini-conferences. Our POS high 
schools — Azure, Sky, and Navy — differed in the intensity and intentionality of their focus on 
project-based learning. At Azure, only a few teachers described project-based learning as a 
hallmark of their curricula; a legal studies teacher noted that although he had participated in 
intensive professional development and built a number of project-based modules into his 
curriculum, the school was no longer financially supporting implementation of the model. At 
Sky, a project-based focus was more evident in some CTE POS than in others; this school 
seemed to place a greater emphasis on work-based learning (e.g., school-based enterprises, 
internships, or off-campus jobs) and the development of 2 1 st -century skills. At Navy, project- 
based learning fonned the core focus of the school’s construction, staffing, curriculum, and 
school-wide learning and service activities. Teachers in Ruby’s magnet POS implemented 
project-based learning with the support of their principal and magnet coordinator. District CTE 
personnel acknowledged that, like work-based learning, project-based learning was more 
challenging to implement in comprehensive high schools without block schedules. However, 
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support for the initiative, particularly from business partners, was quite strong. One such partner 
praised the PLTW curriculum, in particular, for blending engineering technology with 
teamwork-building activities. For this partner, project-based learning offered a means of 
overcoming misperceptions about CTE, better marketing CTE programs, and recruiting high- 
quality students. He also saw it as a powerful tool for creating lifelong learners. 

Project-based learning at Navy. Navy’s principal was instrumental in the design and 
construction of the school, and under his leadership, project-based learning fonned the core 
focus of its physical plant, programs, staffing, and culture. A number of interviewees noted that 
familiarity with project-based learning or willingness to embrace it were key factors in the 
principal’s hiring decisions. All teachers participated in four days of professional development 
each year in addition to intensive summer training sessions. The school also employed certified 
trainers, one of whom served as the school’s project-based learning coordinator. As this 
coordinator noted, although the school’s focus was initially selected by the principal, the 
initiative was fully supported and driven by teachers. Further, Navy’s school-wide project-based 
learning events, such as its solar car challenge, space exploration day, and world cultures 
bazaar — were wildly popular with students, parents, and community members. That popularity 
was borne out by the school’s low dropout rate (especially compared to the comprehensive high 
schools) and high test scores, which the principal attributed to the approach’s focus on teaching 
skills, not teaching to the test: 

If project-based learning was not working, then our scores would tank . . . we’re not 
teaching to the test, but they see the application of the math, the science, the writing, the 
reading, and its use throughout all of our classes. So they’re learning it but not learning it 
for the sake of learning it. They understand the importance of it and I think it shows in 
our scores. 

According to the project-based learning coordinator, this approach to learning was all about 
creating cognitive dissonance that sparks learning — presented with a real-world problem, 
students seek ways to fix it by synthesizing and applying their knowledge and skills: “Let’s 
apply it. Let’s make this real. . . I think that’s the answer specific to this whole school is ‘make it 
real, make them live it.’ If you’re living it, you get it. If you’re regurgitating it, it’s gone. [That 
approach] doesn’t make sense. Never has.” 

The effects of project-based learning were perceived as just as transformative for teachers; as one 
teacher said: “So we give the problem and the students do research to come up with the solution, 
and that sounds real simple in theory, but boy, was that a change for me as a teacher. ... I had to 
let them make the mistake and let them see why it didn’t work. And that’s hard as a teacher - 
real hard.’” 

Curriculum integration was key to the approach. A history teacher explained how his students 
engaged with history through projects that allowed them to explore topics that related to and 
drew upon their POS. In a unit on the Civil War, he collaborated with culinary instructors to 
teach students how to make hardtack, a battlefield dietary staple. Other projects allowed students 
to learn more about local history. In one such project, students worked with the media program 
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to record and preserve oral histories from area veterans; the project culminated in an emotional 
community-wide event hosted and catered by hospitality and culinary students. 

Teaching Soft Skills at the POS High Schools 

West District only used curricula that incorporated state standards. As such, all CTE courses 
included a set of employability skill standards, although the state CTE director believed these 
standards could be better integrated with the other content standards of each course. Below, we 
describe how the POS high schools each took a different global approach to teaching these 
skills — which participants variably referred to as employability skills, soft skills, or 2 1 st -century 
skills. In subsequent sections, we describe how individual soft skills — including team-building, 
critical thinking, problem-solving, and communication skills — were taught in the classroom or 
incorporated in CTSO activities at the POS high schools. 

At Navy, an on-site daycare center offered a real educational setting in which students learned 
how to teach, engage young children in learning and play, write and test lesson plans, and 
develop a year’s worth of curriculum that they could show potential employers or use in college. 
A hospitality teacher noted that the soft skills she emphasized — business writing, customer 
service, and marketing skills — would be valuable even if students chose not to pursue a career in 
hospitality. In planning the many public events that took place at Navy, hospitality students 
prepared guides that described the steps and materials involved in hosting each event; students 
then compared their guides to those produced in previous years, analyzing differences and 
describing how their procedures could be improved. 

Sky developed a 21st-century skills course for ninth graders that included preparing four- year 
graduation plans, researching colleges and careers, writing resumes, making presentations, 
modeling professional dress and grooming habits, and developing a positive work ethic. Beyond 
this course, Sky students had many opportunities to engage in mock interviews with school 
business partners, practice filling out job applications, and create and maintain portfolios of their 
best work to show employers. Students we spoke with expressed gratitude for the internship, 
clinical, and interviewing opportunities they had and considered themselves better prepared than 
their peers at their zoned high schools. 

At Azure, a counselor started a student mentoring program that taught soft skills to ninth-graders. 
Older mentors visited ninth-grade computer classes and led students in fun activities focused on 
decision-making, positive choices, time management, organization, and stress relief. Azure 
teachers found other ways to teach soft skills. One IT teacher stressed the idea of teaching 
students how to collaborate with each other to find infonnation instead of simply asking a 
teacher for help: “We... work together collaboratively, which you’re going to do on the job 
because people are going to know stuff that you don’t, and you’re going to know stuff they 
don’t, so we need to learn to work cooperatively and collectively.” A business teacher invited 
guest speakers, including local university business students, to talk to her class about networking, 
being flexible, and understanding other cultures. This teacher emphasized collegial teamwork 
and competition through a lemonade stand fundraiser in which teacher-created teams of students 
created business plans and vied with each other to generate the most sales. 
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Team-building. Over the course of the study, we observed 35 POS classes and 10 academic 
classes in the POS high schools. In both types of classes, about one-quarter of our observations 
captured instances of in-class teamwork, defined on our observation form as: “working as teams 
on projects or activities” and “more than just working in groups to answer questions at the end of 
the chapter.” In academic classes, we saw students working in groups to respond to the literature 
they read, negotiating their interpretations of the reading as they sought a shared way to produce 
the assigned product (e.g., a play, a drawing). This kind of teamwork required planning, self- 
direction, and imagination. 

In POS classes, teamwork was more varied: We saw students learning to place EKG leads on 
their classmates, baking cookies, surveying their school grounds using theodolites, filming 
scenes that evoked certain moods, and cleaning up the auto shop. In most cases, teachers 
assigned students to teams, and students seemed to be engaged and on task. In one case of off- 
task behavior, we were told that the students exhibiting poor teamwork were late — and forced — 
transfers into the POS and as such lacked the engagement and commitment of their classmates. 

A fine example of teamwork took place at Azure, where groups of students in a Robotics class 
built robots that had to move, pick up a stack of blocks, and load them onto a parallel wooden 
surface. Each team clustered around a kit of parts, with most students taking an active role in 
selecting parts, assembling components, or testing their robots. Although this activity tended to 
promote a great deal of chatter and occasional horseplay, it was hands-on, challenging work, 
with students using all of their communication skills to negotiate solutions together. 

Critical thinking and problem-solving skills. During our observations, we sought to identify 
whether and how students worked with infonnation, conducted disciplined inquiry, and engaged 
in higher order thinking skills (see our observation form in Appendix A, cf. questions III 1 c, IV 1, 
and IV2). The most commonly observed way in which students exhibited critical thinking and 
problem-solving skills was by applying information. Across all types of courses, we saw many 
students using the infonnation and knowledge they had acquired in a prior class, whether they 
were completing worksheets, taking tests, or engaging in hands-on activities. 

We observed critical thinking as students engaged in hands-on activities in their POS classes, 
particularly in their upper-level courses. In introductory courses, concepts were introduced (e.g., 
how to use software), then students applied their knowledge in an assignment demonstrating 
their mastery of the concept. Such assignments tended to be closely prescribed. By contrast, in 
capstone, upper-level classes, teachers stressed longer-tenn assignments with much wider 
parameters. For example, a 3D Animation teacher at Sky asked students to create an instructional 
module that linked academic content from another class of the students’ choosing with advanced 
design principles. One student with an interest in astrophysics created a module about the death 
of a star; another assembled an interactive CD on the culture, food, geography, and people of 
Spain. Another designed a 3D animation that demonstrated how a martini glass dropped from 
various angles and heights would break. These highly sophisticated and attractive projects 
showed that students were thinking deeply not only about the concepts that interested them, but 
also how to draw upon their knowledge of 3D animation and design principles to communicate 
those concepts effectively and memorably. 
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Teaching soft skills through CTSO activities. CTSOs offered valuable opportunities to teach 
soft skills, especially teamwork, leadership, and problem-solving. The POS high schools varied 
in the CTSOs they offered and the extent of their integration with the curriculum. At Navy, some 
CTSOs were more active than others. Marketing students were required to join DEC A because 
the program used some DECA-developed curricular elements; DECA competitions were an 
integral part of the program. Azure had fewer CTSO chapters than Navy or Sky, although one 
teacher leader there received a National Advisor of the Year award from a CTSO, and Azure 
students placed at state CTSO competitions and competed at nationals. 

CTSOs had the strongest presence at Sky, where we saw posters in classrooms and hallways and 
a very large trophy case devoted to the school’s CTSOs in its cafeteria. With broad student 
participation, Sky had a history of national winners in various CTSOs, according to district 
personnel. Sky’s 21st-century skills course also included the first unit of a work readiness 
curriculum developed by a CTSO; the rest of the units were presented in various POS courses as 
a way for students to develop professionally and prepare for competitions. A counselor said that 
students showed great growth in leadership over their four years at Sky as a result of their CTSO 
participation and the many opportunities Sky teachers created to exercise their leadership 
opportunities, self-govern, and “get the job done.” The principal commented that he enjoyed 
seeing students start at Sky as freshmen, running around the halls like “little water bugs;” over 
the years, they came to impress him with their intellectual gifts, professionalism, and poise. 

Sky students also ran the student store, which brought in tens of thousands of dollars that helped 
fund student travel to CTSO competitions. One teacher said that many of the students who went 
to these competitions had never before traveled in an airplane; the skills they learned on these 
trips went far beyond the content of the competitions. 

Teaching Soft Skills at the Comprehensive High Schools 

In a business class at Fuchsia, students worked with a local Toastmasters chapter to improve 
their public speaking skills. Second-year students in this two-year sequence then mentored first- 
year students by helping them format their first speeches and complete time management 
activities. However, this class was not representative of soft skills instruction we observed at the 
comprehensive high schools overall. In our observations and interviews, we saw students 
learning how to complete tasks “the way the boss wanted the job done,” get noticed by taking the 
initiative to do more (e.g., grabbing a broom in an auto shop), plan and prioritize goals, deal with 
machines safely, get along with others, complete jobs on time, and communicate orally and on 
paper so that customers understood the work being done (e.g., writing an estimate for HVAC 
work). The CTE teachers at the comprehensive schools recognized that these lessons were as 
important as the technical skills being taught. 

Team-building. In our observations of 16 CTE classes at the comprehensive high schools (we 
saw no academic classes), about one-quarter captured instances of teamwork. Students appeared 
to enjoy the opportunity to work with one another on assignments, whether these were hands-on 
projects (e.g., taping each other’s wrists in a sports medicine class) or group seatwork. Although 
every instance of teamwork included occasionally off-task or “dead weight” students, such 
students were in the minority — and often already known to the teacher. 
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Critical thinking and problem-solving skills. We observed a particularly strong example of 
problem-solving skills in an advanced 3D Animation class at Dandelion. Students were 
introduced to an open source 3D animation software program, asked to follow its basic tutorial, 
and tasked with creating a three-dimensional pyramid. This activity challenged the students, who 
worked intently for the entire period. Near the end of class, only one student figured out how to 
successfully complete the assignment by creating three-dimensional blocks and tapering and 
stacking them into something resembling a stepped pyramid. 

Teaching soft skills through CTSO activities. CTSOs were popular across West District. The 
automotive, CAD, and photography programs at Amaranth and the medical program at Ruby 
each medaled at state-level competitions in their respective CTSOs. Although most participants 
we spoke with strongly supported CTSOs as a positive aspect of CTE programs, one CTE 
teacher felt that CTSOs sidetracked teachers, making them focus on a subset of their students to 
the detriment of others: “The teacher’s gone and there’s a sub in the room. . . I’ve never been able 
to justify leaving 200 kids and taking three [students] for three days to someplace else.” 

National, State, and/or Local Assessments Provide Ongoing Information on the Extent to 
Which Students Are Attaining the Necessary Knowledge and Skills for Entry Into and 
Advancement in Postsecondary Education and Careers in Their Chosen POS 

When our study began, West’s state had not yet developed technical skills assessments, but by 
the end of 2010, the state CTE director reported that they had begun having discussions 
regarding their development. The state’s goal was to align every program with an industry- 
recognized credential or similar neutral, third-party assessment that demonstrated that students 
had mastered the knowledge base. Graphic design and IT teachers interviewed in 201 1 stated 
that assessments were coming to their programs in the following year. Navy culinary teachers 
helped design their own technical skills assessment, as Navy was a pilot school for these 
assessments. Perhaps because technical skills assessments were not controlled at the state level, 
West CTE teachers did not mention examining assessment data either within or across schools, 
as was the case for West’s core academic teachers. 

Creating the Standards 

Each CTE program area in the state formed a writing committee made up of secondary and 
postsecondary teachers who created or reviewed the standards, course goals, and assessments for 
all secondary programs. These standards then went before the state board of education and were 
presented to all program area faculty for comment. Adherence to the standards was required at 
the secondary level. CTE program and course standards incorporated related state academic 
standards because the state’s goal was for all teachers, not just academic teachers, to be involved 
in helping students pass the high-stakes academic exams. We spoke with many teachers in 
various program areas who had served on these writing committees. 
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New Programs Coming Online 

State mandates required that all new CTE programs be aligned with industry standards. For 
example, the hospitality teachers at Navy who were responsible for revamping their program first 
examined its currently available courses. The teachers disassembled and reconfigured the state’s 
standards for the program into lower-level standards that could serve as the focus of the early 
years of the program and upper-level standards that would allow students to exceed program 
expectations by their senior year. These decisions drove later decisions on the program’s 
curriculum, books, and student projects. 

Navy developed its biotechnology program with the aid of the local university’s medical 
program faculty. Navy’s teacher asked the university faculty whether it would be more beneficial 
to students if her courses approached the topic from a molecular biology perspective or a 
biomedical engineering perspective. The faculty suggested the latter approach, because 
biomedical engineering was a field that students could enter after an undergraduate program, 
whereas molecular biology required postgraduate work. 

Assessments and Standards in the Classroom 

In CTE and POS classes in West District, rubrics were the method of choice for evaluating 
assessment results. Teachers received professional development on rubrics and used online tools 
to create them. A desktop publishing teacher described how she graded her students’ work with a 
rubric of industry-approved standards for layout, alignment, and proximity and repetition of 
graphic items. A media teacher said she used rubrics to judge whether minimum project 
requirements were met; she also used a more global assessment: “Can this be aired?” If the 
answer was no (e.g., the audio was poor quality), students had a chance to improve the project 
and their grade. Although her projects were often completed in groups, she graded individually 
so as to avoid rewarding “freeloaders.” Many teachers said they used rubrics because they 
clarified expectations (e.g., teamwork, content, spelling). One automotive teacher said that his 
rubrics were like a “measuring stick of how well they’ve learned the standard.” He likened the 
rubric to “a competency certificate that you’re using daily.” 

Authentic assessments. Several POS teachers described using authentic assessments. The 
culinary program at Sky lent itself to authentic assessment because the school had a restaurant 
that opened to the public every day. Teachers also told us about building robots that could 
accomplish specific tasks, refurbishing a classic automobile, designing and building a model 
airplane, and developing and teaching early childhood lesson plans. 

The local community college. The quality of West District’s CTE programs varied from school 
to school. For example, many district culinary programs continued to be taught by FACS 
teachers who had no restaurant experience in high school classrooms that resembled home 
kitchens. By contrast, the culinary programs in the POS high schools were aligned with the WCC 
program’s focus on large-scale food production and the culinary arts as a career. Culinary 
classrooms at the POS high school were designed to mimic the types of facilities available in 
commercial kitchens. WCC culinary faculty had begun to work with the district to standardize 
their course offerings across high schools and to assess students’ hands-on skills. 
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A WCC dean said he wanted to institute annual reviews of every career program at every school 
in the district because his college could not maintain its own credibility if it bestowed credits on 
students who could not be said to have legitimately earned them. WCC set up workshops for 
high school faculty during the summer to explain its requirements, but there was no way to 
enforce these requirements with teachers who did not want to change their programs despite 
district-level articulation agreements with WCC. 

During our study, the state began mandating that postsecondary occupational programs use 
industry credentials as assessments for their programs wherever possible. For some of its 
programs, WCC experienced difficulties identifying industry standards or even national 
associations associated with certain industries (e.g., commercial art and graphic design). One 
dean noted that in lieu of industry standards, many of his programs had instituted portfolio 
requirements that allowed open access to introductory courses but required students to provide 
evidence of quality work in order to enroll in more advanced courses. 
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CHAPTER FOUR: East District 

POS as a method of developing and maintaining a skilled workforce in a 
new and changing regional economy 

Abstract 

Like many urban school districts, East District developed multiple magnet and choice programs 
across grade spans. At the high school level, a number of schools (or programs within schools) 
offered a range of magnets available to students through a lottery process. Eighth-grade 
applicants had to demonstrate successful completion of core competency standards in reading, 
math, and computers to be eligible. 

Blue Academy, the wall-to-wall magnet POS high school that participated in this study, served 
students from all over East District. Although Blue held very high expectations for its students, 
teachers, students, and administrators described a supportive, caring culture where barriers to 
success were identified and addressed. 

We planned to test an instrumental variable model of the effects of POS enrollment and CTE 
credit earning on GPA and graduation. However, POS enrollment did not generate substantial 
exogenous variation in number of CTE credits earned, which precluded us from interpreting the 
results of the instrumental variable analysis. We tested a conventional mediator model and found 
that while POS enrollment did not impact CTE credit earning, each additional CTE credit earned 
corresponded to a 9% improvement in the probability of graduation and a . 1 5 unit improvement 
in GPA. These results suggest that in East District, earning CTE credits benefited students in 
POS and comparison schools in terms of retention and also achievement. 

At East District, 50.0% of the intervention group completed a POS. We examined policy- 
relevant questions comparing POS completers to those who completed a CTE concentration, and 
to the rest of the sample. We found that POS completers earned significantly more AP credits 
than the rest of the sample, while students in the rest of the sample accrued more college credits 
than POS completers. 

In assessing the effect of completing a POS on high school achievement through posthoc 
regression analyses, we found that POS completers were more likely than the rest of the sample 
to have earned more AP credits, but completing a POS at Blue did not affect the likelihood of 
earning higher GPAs overall or in CTE, or the likelihood of earning STEM credits. The model 
for dual enrollment predicted a higher number of college credits accrued by the rest of the 
sample, but the results indicated a weak fit to the data, suggesting that important factors in dual 
enrollment participation were not captured in the model. 

Because there was only one intervention school in East District, we compared POS completer 
outcomes to those of CTE concentrators at Blue Academy (i.e., Blue POS students who failed to 
complete their POS but qualified for CTE concentrator status) as opposed to all CTE 
concentrators in the sample. We did this in order to test whether outcomes were attributable to 
completing a POS or if they had more to do with Blue Academy as a school — its reputation, 
culture, faculty, and so on. Compared to the Blue concentrators, completing a POS did not affect 
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the likelihood of earning a higher overall or CTE GPA, or of earning more STEM or AP credits, 
suggesting a school effect rather than a POS effect. The model for dual enrollment showed no 
predictive value to POS completion, but again the fit of the data to the model was weak. 

The senior exit survey at East District showed that students who enrolled in Blue Academy and 
those in the comparison group had very similar post-high school plans: the majority of both 
groups planned to attend a four-year college or university full time, and had already been 
accepted to the school of their choice. Intervention students were more likely to choose a college 
major related to their high school program area than were comparison students who were in a 
specific program in high school (i.e., perfonning arts). 

We then present a qualitative analysis of POS implementation in East District, based on the 
elements of the law mandating POS and the subsequent policy guidance framework provided by 
the funding agency (USDE/OVAE, 2010). 

Introduction to East District and Blue Academy 

We identified East District and Blue Academy as a potential candidate for inclusion in the study 
because of its wall-to-wall technology academy structure and strong, well-articulated POS that 
provided advanced technological training to highly engaged students. 

East District’s state has historically had control over CTE programs in order to maintain a strong 
workforce in the state. Changes to the state economy that mirrored national changes (i.e., a loss 
of manufacturing) led to the retooling of CTE programs and the development of programs with 
more academic rigor and postsecondary opportunities. East District chose to develop high-tech 
POS for their magnet POS high school, which they located near downtown in order to attract 
district students from the immediate suburbs. 

Blue Academy 

Blue Academy is a state-of-the-art high school featuring nine POS in three academies: 
engineering, health careers and biotechnology, and infonnation technology (IT). Blue’s 1,081 
students in 2008-09 reflected the racial and socioeconomic composition of the district; the school 
was predominantly Black, and a large proportion of its students were F/RL-eligible. 

Blue’s structure was marked by strong top-down leadership that was diffused across its three 
academy directors, who undertook such tasks as ensuring that their curriculum was up-to-date 
and that teachers had the resources they needed. The goal of the principal and his leadership 
team was to provide firm, fair, and consistent leadership, a core vision for the school, modeling 
and re-assessment of that vision, trust and collaboration among faculty and staff, and a safe, 
nurturing, challenging environment for students. 

Although national in scope, the recession hit this state particularly hard. The state curtailed the 
ability of students to earn postsecondary credits and take courses at the local community college. 
There were cuts in counseling and support positions, causing teachers to have to take on 
additional responsibilities. Cutbacks also compelled the school to offer more broad-based, 
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introductory courses that enrolled greater numbers of students, and fewer small, specialized 
courses serving more limited numbers of students. Changes in the district’s busing plan also 
created changes in the student body. 

At the comprehensive high schools that made up our comparison schools, the recession cut 
elective funding (including CTE) and staffing levels. Programs and entire high schools were 
closed, causing involuntary student mobility and the interruption of programs. District-wide, 
students were often placed in CTE courses or other electives not of their own choosing. This led 
to students with no interest in a CTE program being enrolled and expected to achieve on the 
states’ end-of-tenn exams. Without motivation, however, achievement suffered. In contrast, 
Blue’s core CTE mission helped preserve programs there, although the elimination of some POS 
capstone courses hurt our cohort’s ability to complete their POS. 

POS at Blue. POS at Blue began in the ninth grade with introductory coursework and a 
mandatory computer class and progressed to more advanced coursework in the remaining three 
years. The POS offered at Blue included five POS in the Engineering Academy: electronics, pre- 
engineering, construction, automotive, and architectural technology. The Health Careers and 
Biotechnology Academy offered two POS, medical and diagnostics and biotechnology. Finally, 
the IT Academy offered POS in IT and business IT. 

How School and POS Culture and Identity Engaged Students 

CTE in East District was a well-supported curricular component in the high schools. 

Instructional technology was in wide use throughout the district, and teachers were clearly 
trained in its use. However, East District was badly affected by the recession, and many of the 
elements of POS suffered. Students at both Blue and the comparison schools lost the state 
funding that waived tuition for them to attend the local community college. Nor could students 
depend on district help in paying for industry-recognized credentialing exams. Such 
opportunities — including some mandated elements of POS — continued to exist, but fewer 
students were able to take advantage of them. 

Blue students enjoyed a school environment that was supportive of CTE as a means to attend 
college, more up-to-date equipment, and many faculty members with recent careers in relevant 
industries. We also found that Blue students appeared to be held to high expectations within a 
culture of support, as described below. 

By contrast, there was no relationship between CTE course sequences and specific academic 
requirements at the comparison schools. These schools often had less well-resourced programs 
that lacked a tangible connection to careers and “the real world.” CTE was much more variable 
in technical and academic rigor at the comparison schools. As in West District, CTE had to 
compete for funds at the comprehensive high schools. Finally, comparison school students could 
take CTE courses in one program area and move to another at any time. 

Blue Academy: A Culture of Caring and High Expectations 
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Culture of caring and collaboration. Blue’s school culture helped the school maintain its focus 
on academic and technical achievement. It emphasized caring and collaboration, high 
expectations, and college and career planning as part of its daily activities. This culture was 
communicated on multiple fronts using a variety of media, including posters in hallways, 
mandatory student schedule books, and closed-circuit monitors in each classroom that delivered 
a constant stream of interesting facts, school news, and inspirational quotes. 

The principal introduced a corporate-inspired philosophy about implementing change through 
personal responsibility and positive interactions with others. Mascots embodying this philosophy 
were displayed in classrooms, hallways, and offices, and posters featuring its main tenets hung in 
common spaces. The principal told us that he thought students had bought into it; he saw this 
reflected in the older students’ practice of befriending ninth graders, who, coming to Blue from 
far-flung comers of the district, were unlikely to know other students. We also observed students 
in a lOth-grade English course critiquing each other’s work. They were required to comment on 
their partner’s writing or the student being assessed would lose points. In this way, students 
knew their responsibility for one another’s learning: the teacher reminded them to respond “out 
of respect for them.” 

High expectations. High expectations were visible in the staffs frequent monitoring of student 
progress and sponsorship of friendly competitions to spur higher achievement. One teacher had 
students from each of his classes post their self-determined goals (i.e., increase end-of-tenn test 
scores by two points) on the wall for others to see, along with graphs of their progress. 

College and career planning. College-going was championed at Blue — for example, a banner 
hung in one classroom read, “Students who are the first in their families to go to college END 
poverty in their family lines forever. - U.S. Department of Labor.” College and career planning 
was suffused in the classroom, through the curriculum and through many college- and career- 
oriented assemblies and field trips. At the end of one biotechnology class we observed, the 
teacher spoke of an upcoming field trip to a biotechnology company in the area. She related a 
story about a fonner student who during a tour the previous year caught the attention of company 
staff; this student interviewed with the company and got a job upon graduation. 

Blue’s principal stated that his philosophy was not “to graduate them from high school. I want 
them graduating from college. So I’m pushing - I push a lot more if the AP courses match up 
with the CTE courses - I need them to participate in the most rigorous courses available to them 
so that they are going to college better prepared.” This principal and East’s CTE director spoke 
of raising the level of expectations, and how students had risen to it. These educators knew that 
supports had to be in place for that to happen, and they were in place at Blue: teachers were paid 
to stay after school or on Saturdays to tutor students or prepare them for college entrance exams. 

Instrumental Variable Sample Description 

The East District portion of the study examined data for 1,128 secondary students; 376 enrolled 
in Blue Academy, the POS high school, in ninth grade, and 752 attended other schools. Table 4.1 
displays descriptive statistics for the East District imputed sample. 
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Table 4.1 


Descriptive statistics for imputed sample, East District 



(N= 1,128) 

% 

Overage 



No 

1,113 

98.7 

Yes 

15 

1.3 

Gender 



Male 

571 

50.6 

Female 

557 

49.4 

Race/Ethnicity 



Black 

797 

70.7 

White 

124 

11.0 

Latino 

119 

10.5 

Asian 

41 

3.6 

Native American 

2 

0.2 

Other 

45 

4.0 

F/RL eligible 

756 

67.0 

Limited English Proficient (LEP) 

52 

4.6 

Has Individual Education Plan (IEP) 

26 

2.3 

Graduated on time (2012) 

861 

76.3 

Attrition 

207 

18.4 


Mean 

SD 

Discipline incidents 

0.54 

1.31 

Grade 8 pre-test math scale score 3 

363.00 

7.35 

Grade 8 pre-test reading scale score 3 

359.76 

7.36 

Grade 8 pre-test science scale score 3 

150.07 

7.86 

Number of GTE credits earned 

5.24 

2.01 


Note. F/RL = free/reduced price lunch program. 
a Variables were grand mean centered. 


Missing Data 

We carried out a missing data analysis for East and observed that the percentage of data missing 
for the variables ranged from 0.0 to 35.3%, with an average of 2.7%. As with West District, we 
rejected the assumption of MCAR given mean differences in eighth- grade standardized test 
scores between those who were and were not missing GPA data for later years. We made the 
more relaxed assumption of MAR and handled missing data using the same procedure as 
reported for West District. 


Instrumental Variable Estimation 

We tested our hypothesized model across the imputed sets and observed that it was over- 
identified with 13 degrees of freedom and fit the data poorly (e.g., % = 141.26, CFI = .88, TLI = 
.58, and RMSEA = .09). We tested the model using a single imputed set to obtain a global test of 
fit and modification indices. We observed a significant y test statistic (p < .001) and 
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modification indices and standardized residuals indicating that number of CTE credits earned 
should be regressed on LEP status, discipline referrals, Latino, Asian, Black, and Grade 8 math 
score. We added these specifications and observed that the resulting model was over-identified 
with 8 degrees of freedom and fit to the data was excellent (e.g., "i = 14.47, CFI = .99, TLI = 

.97, and RMSEA = .03). We tested the re-specified model using a single imputed set and the 
global test statistic was non-significant ip = .07), suggesting we should not reject the hypothesis 
that model fit the data exactly (see Bollen, 1987, pp. 263 - 269). 

Enrollment in Blue Academy appeared to increase the number of CTE credits students took by 
an average of .19 credits, explaining less than 1% of the variance in CTE credits earned. 
Controlling for background variables, the indirect effect of enrolling in Blue on GPA was non- 
significant, and so was its indirect effect on graduation. The direct effect of enrollment in Blue 
Academy on number of CTE credits was small but significant (i.e., b = .19, p < .01, (3 = .05). The 
direct effects of CTE credits on GPA and graduation were non-significant. We noted a non- 
significant residual covariance between number of CTE credits and GPA. However, the residual 
association between number of CTE credits and graduation was large and observed at cov = 2.32 
ip < .001). Taken together, these results imply that in this district, enrollment in the POS school 
was not a good instrument and failed to generate adequate exogenous variation in number of 
CTE credits earned. Indeed, attendance at Blue Academy only appeared to increase the number 
of CTE credits earned by . 19 and explained a trivial amount of its variance, while in the West 
District model, enrollment in their POS schools increased the number of credits earned by 2.82 
and explained 28% of the variance in credits earned. Because weak instruments tend to produce 
unstable estimates, we did not interpret the results for this model. 

Given the findings above, we tested a conventional mediator model by removing the covariances 
connecting the graduation and GPA residuals to the number of CTE credits earned residual. This 
model was over-identified with 9 degrees of freedom and fit the data very well (e.g., % = 13.55, 
CFI = .99, TLI = .98, and RMSEA of .02). We tested the model using a single imputed set to 
obtain a % test and modification indices. We observed a statistically significant global test 
statistic {p = .14), suggesting that we should did not reject the hypothesis that the model fit the 
data exactly. 

For this model (see Figure 4. 1), enrollment in Blue Academy was not a significant predictor of 
number of CTE credits earned. The direct effect between CTE credits and GPA was b = .15 ip < 
.001, (3 = .34), while the direct effect between credits earned and graduation was b = .45 ip < 
.001). Taken together, these results indicate that students who enrolled in Blue were not any 
different than those who enrolled elsewhere in terms of CTE credit earning. However, for the 
sample as a whole, earning one additional CTE credit was associated with a 9% greater 
probability of graduation. In addition, the standardized effect between CTE credits earned and 
GPA was moderate in size ([3 = .34), and each additional CTE credit earned corresponded to a 
.15 unit increase in GPA. Although student CTE credit earning behavior was not changed by 
enrollment in the POS school at East, the data suggest that earning more CTE credits benefited 
students. We note that this model did not correct for endogeneity so causal inference must 
remain tentative. 
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In addition to the effects described above, several covariates were significant predictors of 
number of CTE credits earned, GPA, and graduation. Gender (i.e., being male), overage, F/RL, 
discipline occurrences, and Latino had significant and small negative effects on GPA, while 
Grade 8 grade science and reading scores had small but significant positive effects on GPA (ps < 
.05). In addition, Grade 8 grade math score had a significant moderate positive effect on GPA (p 
< .001). Gender, F/RL, special education, discipline occurrences, and Grade 8 grade science 
score had significant and small negative effects on graduation, while Black and Grade 8 grade 
math score had significant and small positive effects on graduation (ps < .05). Finally, LEP and 
discipline occurrences had significant and small negative effects on number of CTE credits 
earned, while Latino, Asian, Black, and Grade 8 grade math score had small but significant 
positive effects on number of CTE credits earned (ps < .001). For a complete description of the 
estimates for East District, see Appendix A. 

Figure 4.1 

Final model for East District 


Z 

(Enrolled in 
POS) 



Note: Significant unstandardized estimates are displayed. Covariates are omitted for conceptual clarity. 

Assessing the Effect of Completing a POS on High School Achievement 

Our second approach to data analysis employed multiple regression analyses, using a different 
sample — POS completers, all other students, and CTE concentrators. We excluded students who 
(1) withdrew or were missing course data and (2) were missing baseline achievement data. We 
were consistent with this so that the regression sample would match the descriptive statistics to 
the extent possible. The sample for the CTE GPA analysis is further reduced to those who 
actually took a CTE course among the student in the All Others group. Table 4.2 provides 
descriptive statistics for the posthoc regression sample. 
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Table 4.2 


Descriptive statistics, posthoc sample, East District 


Total 

POS 

All Others 

Blue CTE 

POS 

POS 

(n = 907) 

Completers 

(n = 720) 

Concentrators 

Completers 

Completers 


r~7 

00 

II 


(n = 39) 

Compared 

Compared to 





to All 

Blue CTE 





Others 

Concentrators 


Characteristics 


Male 

49.5 

43.9 

51.0 

74.0 

Black 

71.0 

75.9 

69.7 

66.7 

White 

10.6 

8.0 

11.3 

12.8 

Latino 

11.1 

10.7 

11.3 

7.7 

Asian 

3.9 

3.2 

4.0 

7.7 

Other/Multiracial 

3.4 

2.1 

3.8 

5.1 

F/RL eligible 

65.8 

67.9 

65.3 

59.0 

LEP 

4.2 

2.7 

4.6 

2.6 

IEP 

1.1 

1.1 

2.1 

2.6 

Grade 8 pre-test reading 

360.19 (7.10) 

360.74 (7.16) 

360.04 (7.08) 

358.31 (6.14) 

Grade 8 pre-test math 

363.43 (7.22) 

363.83 (7.11) 

363.43 (7.22) 

362.72 (5.82) 

Discipline events 

None 

79.4 

79.1 

79.4 

79.5 

1-2 events 

16.5 

17.1 

16.4 

17.9 

3+ events 

4.1 

3.7 

4.1 

2.6 


Note. Excludes students who withdrew or were missing course data. Standard deviations of continuous variables are included in parentheses. 
*p < .05. 
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Table 4.3 


Descriptive outcome statistics, posthoc sample, East District 



Total 
(n = 907) 

POS 

Completers 
(n =187) 

All Others 
(n = 720) 

Blue CTE 
Concentrators 

0 n = 39) 

POS 

Completers 
Compared 
to All 
Others 

POS 

Completers 
Compared to 
Blue CTE 
Concentrators 

Percent Earning Outcome Credits 






STEM 

99.6 

100.0 

99.4 

100.0 



AP 

36.8 

55.1 

32.1 

38.5 

*** 


College (accrued) 

5.2 

3.7 

5.6 

2.6 



CTE 

94.3 

100.0 

92.8 

100.0 



Outcome Means 

M(SD) 

M(SD ) 

M(SD ) 

M(SD ) 



Overall GPA 

2.69 (0.74) 

2.77 (0.65) 

2.67 (0.76) 

2.64 (0.63) 



STEM credits earned 

5.12(1.43) 

5.29(1.26) 

5.08 (1.47) 

5.00 (1.28) 



AP credits earned 

1.18(2.04) 

2.08 (2.55) 

0.95 (1.81) 

1.62 (2.63) 



College credits accrued 

0.80 (4.53) 

0.19(1.04) 

0.97 (5.04) 

0.08 (0.48) 

*** 


CTE GPA 

2.76 (0.83) 

2.79 (0.68) 

2.75 (0.87) 

2.65 (0.72) 




n = 859 


n = 672 





Note. Excludes students who withdrew or were missing course data. STEM credits equals credits in science and higher math courses (i.e., above Algebra II). 
College credits accrued refers to the potential college credits associated with dual enrollment participation in high school. 

"Vc.OOl. 
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We found that of the 376 East intervention students, 188 (50.0%) completed a POS. We then 
identified the number of students who earned credits in the outcome measures defined in the 
Method chapter (i.e., measures of academic rigor, technical rigor, and transition as outlined 
there). Descriptive results of student course-taking are shown in Table 4.3. 

For the comparison of POS completers and the rest of the sample, the results for AP credits 
earned were positive, statistically significant, and have an effect size of 0.52, making this the 
most substantively important finding at East District (cf. Table 4.4). The dual enrollment finding 
is significant favoring the rest of the sample, but due to the low effect size and poor fit of the 
model to the data, the finding is questionable. 

Table 4.4 

Regression results, POS completers versus all others, East District 

East District POS Completers Versus All Others 


Outcome 

n 

Adjusted 

R 2 

b 

SEb 

fi 

P 

95% Cl of b 
lower upper 

ES 

Overall GPA 

STEM 

credits 

907 

0.523 

.054 

.042 

.030 

.196 

-.028 

.137 

0.07 

earned 

AP credits 

907 

0.342 

.141 

.096 

.040 

.140 

-.046 

.329 

0.10 

earned 

College 

907 

0.348 

1.067 

.136 

.212 

.000 

.801 

1.333 

0.52 

credits 

accrued 

907 

0.023 

-.846 

.369 

-.076 

.022 

-1.571 

-.121 

-0.19 

CTE GPA 

907 

0.139 

.027 

.064 

.014 

.667 

-.098 

.153 

0.02 


Note. All coefficients rounded to two digits. Full regression tables are found in Appendix A. CI= Confidence 
Interval. ES = effect size (Hedges’s g) 


Table 4.5 shows the comparison of POS completers and Blue CTE concentrators. There were no 
statistically significant results; however, for CTE GPA, there is a substantive positive effect size, 
meaning that although the finding did not reach statistical significance, there is practical 
significance in the number of CTE credits earned by POS completers compared to CTE 
concentrators at Blue. 


Senior Exit Survey Findings 

Sample students at East District were given a senior exit survey in May 2012 (see Method). 
Response rates were not high. Significantly more intervention students took the survey than 
comparison students (42.0% versus 32.2%, p = .001). 

The intervention group’s survey respondents were not significantly different from the 
intervention nonrespondents. Among the comparison group students, significantly more Black 
and Asian comparison students took the survey than not, and significantly fewer White 
comparison students and LEP comparison students took the survey than not (see Appendix D ). 
All of these differences should be taken into account when interpreting the survey results. 
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Intervention and comparison survey respondents were also compared to each other, and no 
significant differences were found, as shown in Appendix D. 

Table 4.5 

Regression results, POS completers versus Blue CTE concentrators, East District 

East District POS Completers Versus Blue CTE Concentrators 


Outcome 

n 

Adjusted 

R 2 

b 

SEb 

fi 

P 

95% Cl of b 
lower upper 

ES 

Overall GPA 

STEM 

credits 

226 

0.463 

-.012 

.088 

-.007 

.887 

so 

OO 

.161 

-0.05 

earned 

AP credits 

226 

0.372 

.107 

.183 

.032 

.559 

-.254 

.469 

0.06 

earned 

College 

226 

0.401 

■ 

o 

'o 

.366 

-.016 

.766 

-.830 

.612 

-0.08 

credits 

accrued 

226 

0.040 

.225 

.175 

.088 

.201 

-.120 

.569 

0.19 

CTE GPA 

226 

0.096 

.202 

.119 

.111 

.090 

-.032 

.437 

0.25 


Note. All coefficients rounded to two digits. Full regression tables are found in Appendix A. Cl = Confidence 
Interval. ES = effect size (Hedges’s g) 


Table 4.6 provides information on students’ next steps after high school graduation. The students 
who responded to the survey were very similar in their post-high school plans, regardless of 
whether they were in the intervention or comparison group: most had been accepted (over 78% 
of the sample) and planned to attend a four-year college or university (over 71% of the sample) 
full time (about 97% of the sample) in Fall 2012. There were no significant differences in their 
work plans either: most respondents planned to work part time, and those jobs were 
predominantly not related to their high school program. Only a small percentage planned to join 
the military, but relatively more intervention students reported that as their next step than 
comparison students. 

We asked students whether their postsecondary studies would be related to their high school 
program. Of those students who indicated that they were in a high school program (defined in the 
survey as “major, program of study, career major, career pathway, career academy, International 
Baccalaureate, etc.”), 69 percent of the intervention students indicated that their post-high school 
program would related to their high school program. This is similar to the comparison group 
percentage who indicated that their post-high school studies would relate to their high school 
program (64%). One possible reason for the similarity in response could be that one of the 
comparison schools at East was a perfonning arts high school. Many of those students would 
have self-identified as having participated in a program in high school, and many might have 
also been planning to continue their arts program in college. In either case, the results suggest 
that participating in an identifiable high school program can help clarify students’ post -high 
school planning. 
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Table 4.6 


Student exit survey responses, East District 


Survey Items 

Intervention 
(N= 152) 
(%) 

Comparison 
(A =233) 
(%) 

Do you plan on working immediately after graduation? 

Yes 

59.9 

51.1 

Will you be employed full time? 

Yes 

46.2 

35.3 

Is this job related to your high school program? 

Yes 

13.2 

9.2 

No 

71.4 

62.2 

Not in a program 

15.4 

28.6 

Do you plan to attend a four- year college or university, 
two-year community college, or trade/ technical school 
for the Fall 2012 semester? 

Yes 

88.8 

93.1 

Four-year 

74.8 

71.8 

Two-year 

21.5 

25.5 

Trade/technical 

1.5 

1.4 

Other 

2.2 

1.4 

Have you been accepted to this school? 

Yes 

79.3 

78.2 

Will you be a full-time student? 

Yes 

97.8 

96.8 


Will your studies at this school relate to your high 
school program? 


Yes 

63.0 

44 4 ** 

No 

28.9 

25.5 

Not in a program 

8.1 

30.1*** 

Have you been accepted into a branch of the military? 

7.2 

4.3 


Note. Due to missing data, percentages may not add up to 100. Significance levels are based on chi-square analyses 
comparing intervention and comparison groups. 

*p < .05. **p < .01. ***p < .001. 


Adherence to the Components of POS 

East District offered vertically aligned POS designed by secondary and postsecondary educators 
in conjunction with business and industry partners; POS curricula, standards, and assessments 
were either generated at the state level or locally developed and submitted for state approval. For 
students in our cohort, East’s POS were comprised of four courses in a progressively intensive 
sequence, culminating in a capstone course and coupled with opportunities to earn articulated 
college credits. Although East certainly stressed the career aspect of college and career readiness, 
in part by embedding career development and exploration in all of its CTE courses, college 
credit-earning, and college preparation in general, were a major press in this district. The district 
greatly benefited not only from its partnership with a highly regarded local community college, 
but also from the state’s efforts to establish articulation agreements between its secondary 
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education agencies and its community college and university systems. Although committed to 
offering students a broad range of college credit-earning opportunities, the recession challenged 
the district’s ability to do so; as such, over the course of our study, it increasingly emphasized 
lower-cost online coursework. Similarly, although industry-recognized credentials were a fairly 
low policy priority, the recession limited the district’s ability to offer such credentials, or to pay 
for students to take credentialing exams, in the programs in which they were available. 

The recession’s impact on East was grave and far-reaching, and greatly affected its ability to 
implement and sustain many elements of POS. The district endured personnel cuts (teachers, 
administrators, and support positions), school and program closures and reorganizations, drastic 
transportation reductions, and the limitations of aging and inadequate technology. Despite such 
challenges, the district remained committed to supporting its CTE programs with the resources, 
personnel, and training those personnel needed to offer high-quality educational experiences. 
Business and industry partners offered material, financial, and personal assistance to help the 
district achieve this goal, including by providing opportunities for students to participate in 
work-based learning experiences (i.e., internships) and donating much-needed equipment. 

Both core academic course assessments and CTE end-of-term assessments were managed at the 
state level and administered via a fairly recently adopted online assessment system; CTE 
assessments were developed by educators with input from industry, or were associated with 
industry-recognized credentials. A major component of this online system was its integrated data 
analysis tools, which allowed educators to both track and predict students’ performance over 
time. Many academic and CTE teachers described receiving training to help them adapt to this 
system; teachers also mentioned collaborating with each other in professional learning 
communities in which they used systems data to target student deficiencies and improve their 
instruction. For POS and CTE programs, a major weakness of this system was its lack of a 
perfonnance-based component. 

Interdisciplinary teacher collaboration and curriculum integration were most often observed at 
Blue, where block scheduling and the schooFs academy structure made such instructional 
approaches a more natural fit. We saw fewer examples of either approach at our comparison 
sites. Project-based learning was not a major district initiative; however, we observed more 
examples of it at Blue than at the comparison sites. In both Blue’s POS and regular CTE 
programs, workplace readiness skills were infused in classroom instruction and CTSO activities. 

Incorporate and Align Secondary and Postsecondary Education Elements 

East District is located in a state considered at the forefront of providing standards-based CTE 
and seamless transitions from secondary to postsecondary education. The state designs and 
maintains articulation agreements with postsecondary institutions, although local education 
agencies may also develop their own local agreements. Postsecondary faculty members provide 
input into secondary CTE curricula, standards, and assessments, usually at the state level. 

At the comparison schools, participants reported somewhat limited direct contacts with 
postsecondary. A digital media teacher at Heliotrope reported that district CTE leadership, rather 
than teachers, worked with East Community College (ECC) staff in his program area. This 
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teacher was aware that ECC faculty designed some of the activities he used in his class. He also 
reported that ECC IT faculty members were working with East District middle and high schools 
to increase interest in IT careers among girls and minority students. 

At Blue Academy, in contrast, an IT teacher described a much closer relationship with ECC. 
Faculty members from both ECC and the state university system sat on the advisory boards of 
some of Blue’s programs. This teacher also described how the district IT curriculum alignment 
committee met monthly with the college programming instructor in attendance in order to ensure 
that all of the high school programming teachers were on pace with their shared syllabus. This 
committee had begun to meet at ECC. New agenda items included aligning and negotiating 
program content and appropriate student entry into the postsecondary part of the program. 

Prior to our study, ECC developed a strong partnership with Blue Academy that included 
certifying Blue teachers to teach postsecondary-level courses and giving Blue students 
opportunities to attend after-school clubs and take courses offered by ECC instructors. One 
teacher described a defunct program in which Blue students took classes on the ECC campus: 

They could leave Blue and the district would pay for them to go to ECC and take game 
simulation. We would give the kids a bus pass that couldn’t afford one. But they would 
get on the bus in front of the school, go to ECC, and take their night class, and they were 
excused from us. And then some of them that work, that was great for them because they 
could earn a little money, they earned college credit, and they’re still in the high school - 
can you imagine the independence that they felt and you know, the parents are looking at 
the responsibilities that they have, and the maturity level, you know, they can drive. I 
mean it was great. Kids loved it. 

Post-recession, the grants and funding to support such programs and opportunities disappeared. 
Although interviewees at ECC lamented the loss of these funds and the opportunities they 
provided, they had mixed feelings regarding the return on their investment in Blue. Participants 
believed that despite the college’s generous underwriting of the faculty time and resources 
needed to partner with Blue’s programs, Blue faculty were encouraging their students to apply to 
universities, not the college’s highly regarded and highly accessible programs: “Their students 
over there - they’re pushing them towards the four-year schools.” 

As our data collection efforts in East drew to a close, additional alignments between district and 
ECC curricula were being cultivated. For example, East was in the process of implementing 
PLTW programs in high schools all over the district. To better align with this curriculum, ECC 
proposed to develop a pathway from the district’s high school-level pre-engineering programs to 
related programs in its department of Engineering Technology. The college’s plan was to better 
align its program prerequisites with the end-of-term exams given in PLTW courses. ECC also 
proposed to cohort any graduates of high school PLTW programs who enrolled at the college. 
East’s CTE department proposed to begin recruiting students for this aligned secondary-to- 
postsecondary pathway in district middle schools offering PLTW’s Gateway to Technology 
program. The district’s expansion of PLTW also sparked deeper conversations with the state 
university system, which had previously done little recruiting in the district. Indeed, the chair of 
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the engineering department noted that there were no district graduates in his program at the state 
university campus located in the heart of East District. 

Professional development offered another means through which elements of postsecondary 
education were incorporated into secondary programs. East District counselors and career 
coordinators received training from ECC staff on the occupational programs the college offered 
and the grants and scholarships available to students. 

In addition to ECC, the state and district also maintained partnerships with other postsecondary 
institutions. For example, a state university hosted summer enrichment programs for high school 
students, including in engineering and architecture, for which students could earn college credits. 
Another community college located in a different county developed a curriculum used by Blue’s 
biotechnology program; teachers in Blue’s program took their students to see lab experiments in 
this college’s facilities. Proprietary schools also worked with the district; in one example, a local 
trade and technical college offered scholarships to students interested in completing their senior 
year at the college while simultaneously beginning their postsecondary coursework. 

Include Academic and CTE Content in a Coordinated, 

Non-Duplicative Progression of Courses 

East’s approach to POS course sequencing and secondary-postsecondary alignment evolved in 
response to changes to state -mandated graduation requirements. Students in the Class of 2012, 
our study cohort, began high school when district students had a choice of four graduation 
options involving varying emphases on college-preparatory and career-preparatory coursework. 
For CTE students, four courses in a defined sequence, with one upper-level or capstone course, 
were required to be considered a CTE completer. Students could choose to concentrate in areas 
linked to about a dozen of the 16 Career Clusters. Early in the study, however, East’s approach to 
course sequences changed to embrace what the district’s CTE director and other participants 
described as a “Clusters model” that de-emphasized more narrowly defined pathways and sought 
to free students to design their own programs and gain a “broader understanding of careers.” 
Sequences were envisioned as more of a “funnel” than a “pipeline,” beginning with a common 
entry point — broad introductory courses within each cluster — and opening up to a range of 
higher-level courses. For example, a student might begin a course of study with a Principles of 
Business course, then go on to take additional courses like Accounting or Computer Applications 
before completing a final, upper-level class. The district saw this model as a way to streamline 
course offerings, consolidate advisory committees, better market CTE, and eliminate the burden 
of maintaining and updating pathways. The recession made these aims more desirable. 

Defined four-course sequences remained viable and sustainable at Blue Academy, in magnet 
programs based at the comprehensive high schools (e.g., Emerald’s burgeoning NAF Academy 
of Engineering offering a PLTW curriculum), and in high schools offering PLTW. Indeed, the 
district’s CTE director considered PLTW to be the district’s only true POS outside of Blue. 
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The Local Community College 


ECC department chairs, faculty, and counselors actively participated in the district’s efforts to 
streamline its course sequences. An IT department chair described how the college worked with 
the district to eliminate restrictive pathways and duplicative programs in order to create: 

One integrated technology degree [that] will have a certain set of core courses, just like 
we have now. But then they’ll have tracks out of it and the students will actually take a 
diploma [that] will identify their track. And so we’ll go back to one program, but multiple 
tracks, instead of having a lot of separate programs. That will make it much easier for a 
student to come in — everybody will know what the core has to be — and then you can 
take these other tracks and if you get into one of those tracks and you decide you don’t 
like it, you can probably use everything you’ve taken as an elective in some others so that 
you don’t lose the credits. . . we tried to make it as flexible as possible and yet still let the 
student specialize. 

ECC counselors spoke of their efforts to align curricula and standards with the state and district 
as creating prescribed pathways in the humanities and social sciences or engineering and 
mathematics that ran through “13th grade” (e.g., four years of aligned high school coursework 
plus a first year of community college). College faculty believed that these alignment efforts 
would help create a pipeline of talented students for their programs. 

N on-Duplicative Sequences at Blue Academy 

As the state and district moved to implement a looser, clusters-based interpretation of POS, 
students in our cohort faced challenges in completing their more prescribed course sequences. At 
the beginning of the study, Blue offered postsecondary-aligned POS linked to a number of 
industry-recognized credentials, as described later in this report. The recession forced Blue to 
close some of its programs and eliminate a number of upper-level courses with low enrollments. 
Such cutbacks, coupled with the district’s defunding of early college learning opportunities, 
compelled some students in our cohort to fill their schedules with courses outside of their 
prescribed programs. Our transcript analyses, completed after our cohort graduated in May of 
2012, revealed that whether due to poor advising or a simple lack of available options, many 
students in our cohort failed to complete their POS under the old graduation requirements to 
which they were still subject. 

Changes in the administration at Blue also impacted the availability of POS programs and 
courses. The first principal we interviewed, who left Blue after the study’s second year, 
understood that the POS available at Blue differed from the CTE options at the comprehensive 
high schools. This principal, who instituted the school’s dedicated academy coordinators and 
counselors, encouraged students to enrich their experiences by taking courses offered in any of 
the school’s other POS once they had completed their own sequences. The school’s new 
principal stated that he valued the way in which the school’s academies made learning relevant, 
integrated academic and technical content, and prepared students to be lifelong learners. 
However, the new principal also referred to certain programs (including automotive, which he 
closed) as “dinosaurs” that held no appeal for on-grade level (i.e., higher-achieving) students. 
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Non-Duplicative Sequences at the Comparison Sites 

The availability of non-duplicative CTE sequences at other district high schools varied according 
to the amount of CTE these schools offered. Some of our comparison sites, like Neon, the 
district’s arts magnet, offered very little CTE and hence no POS sequences that met the Perkins 
IV definition. Other schools offered a broader range of CTE programs with enough introductory, 
mid-level, and capstone courses to allow students to complete full sequences. The 
comprehensives also varied in the extent to which their programs offered industry credentials or 
certificates or credit transfer opportunities. Among our comparison sites, Emerald’s sector-driven 
clusters in engineering, automotive, health, and marketing stood out. In the last two years of our 
study, Emerald’s CTE and academic teachers were engaged in the process of designing and 
implementing a sequence of integrated, STEM-focused courses for its new NAF engineering 
academy offering a PLTW curriculum. 

Offer the Opportunity, Where Appropriate, for 
Secondary Students to Acquire Postsecondary Credits 


Dual Enrollment 

This state maintained control over much of the alignment between high school and 
postsecondary education. Dual enrollment and articulation agreements were required statewide, 
although local education agencies could tailor these agreements. Related to dual enrollment, the 
state mandated that universities and community colleges offer tuition-free, pre-approved college- 
level classes to high school students aged 16 or over. Students had to have a 3.0 GPA and pass a 
college readiness exam (e.g., ACT, SAT, PSAT). Eligible students could earn high school credit 
and start a college transcript. One principal called dual enrollment “better than taking AP, 
because you actually have credit on a transcript.” 

Scheduling conflicts between the district and ECC made it difficult for East students to take ECC 
classes. Lack of transportation was also a barrier to participation. Due to issues like these, East 
made online postsecondary course-taking a major component of its dual enrollment offerings. 
High school juniors and seniors could enroll in selected online college courses that were often 
offered in structured sessions in their zoned high school’s media center. A Blue administrator 
believed that offering online college courses as regularly scheduled classes, overseen by school 
personnel, helped students achieve success. Indeed, staff at schools that did not offer structured 
online classes frequently talked about students struggling with or dropping their online courses. 

Although the state had previously approved postsecondary general education and humanities 
course-taking, the recession forced it to limit its dual enrollment offerings to STEM and CTE 
courses (e.g., Anatomy and Physiology, Java Programming). Students could not take courses that 
were also offered at their high school, but all options remained available to students in online 
courses. 
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Articulation Agreements 

Through both local and statewide articulation agreements, students in approved high school CTE 
courses, usually at the mid- or capstone level, were eligible to receive tuition-free college credit 
based on their end-of-term exam performance and course grades. A statewide set of articulation 
agreements between the state community colleges and the university system allowed students to 
potentially earn what ECC counselors called a “Triple Crown” of high school, community 
college, and university transfer credits. 

All applicants to ECC were required to show college counselors their high school transcripts at 
registration, at which time any articulated courses for which they were qualified to receive 
credits were taken into account. Students could also ask ECC department chairs to review their 
transcripts, but a number of ECC faculty in the college’s engineering and machining programs 
with whom we spoke reported never having done so. 

Students learned about available credit transfer opportunities from ECC representatives who 
visited district high schools three times a year (in the fall, in the spring, and to register seniors). 

Blue personnel appeared well-informed of the credit transfer opportunities available to their 
students. Blue’s IT Academy Coordinator noted that because college faculty wanted to attract 
top-quality students to their schools, she was also able to negotiate college credit for IT students 
with industry certifications. 

At the comparison schools, some administrators and CTE teachers we interviewed were not as 
knowledgeable about available credit transfer opportunities. Some did not know whether their 
students took advantage of these opportunities, or how many did so. Interviewed comparison 
students did appear aware of these programs and could cite their requirements. By contrast, 
interviewed Blue students seemed more familiar with AP credits than articulated credits. 

Not all students actually obtained the articulated credits they accrued in high school. In some 
cases, students were required to pass an additional challenge exam and did not do so. In other 
cases, students enrolled in ECC with the declared intention to transfer to a four-year university, 
in which case the occupational credits they had accrued — because they would not count toward 
transfer — were unlikely to be granted. As one administrator noted, some students saw themselves 
as “four-year students” and did not value the credits: Some ECC occupational credits did not 
transfer to out-of-state schools or in-state universities. One high school guidance counselor told 
us that he encouraged his students to take online university courses instead of articulated 
coursework because of the risk that earned ECC credits would be thrown away. 

Lead to an Industry-Recognized Credential or Certificate at the 
Postsecondary Level, or an Associate or Baccalaureate Degree 

Of the three districts participating in the study, East appeared to offer the fewest opportunities for 
students to earn industry-recognized credentials. A number of East schools featured CTE 
programs with built-in credential or certificate-earning potential, such as CISCO, NAF, NATEF, 
NCCER, or PLTW. Such programs were found at Blue, our POS high school, and at Emerald, a 
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comprehensive high school with a developing NAF engineering academy. Although a number of 
participants noted that constrained resources hampered their ability to offer industry-recognized 
credentials, it was also the case that industry credentials were simply not a major district policy 
focus. Instead, district CTE personnel described the use of NAF academy completer certificates 
for students who completed such programs. District personnel acknowledged that it was difficult 
to get people from industry or postsecondary institutions to recognize the value of the NAF 
academy completer certificates compared to industry-recognized credentials. 

District students were also eligible to receive three varieties of state career readiness certificates, 
derived from ACT’s WorkKeys tests, as an indicator of basic work readiness. Both district CTE 
personnel and ECC counselors cited these certificates as being recognized by business and 
postsecondary. However, only a few interviewed students seemed aware of the availability of 
these certificates, or other credentials for that matter. 

Available Credentials 

Blue’s three academies (Engineering, IT, and Medical and Biotechnology) offered a range of 
POS and a limited number of industry-recognized credentials. In Engineering, these included an 
automotive POS based on NATEF standards that prepared students to take the ASE examination; 
an NCCER-accredited construction program; and a PLTW-accredited pre-engineering program. 
Students in the Engineering academy were encouraged to get their OSHA certification. The IT 
academy’s CISCO program offered the Cisco Certified Entry Networking Technician (CCENT) 
certification, CompTIA A+, C++, Oracle, and SAS. The Medical and Biotechnology academy 
was nationally recognized, but offered no formal industry credential; instead, the medical POS 
prepared students to take the CNA exam at their own expense elsewhere. According to the 
former head of the academy, the academy’s advisory board was working with a local 
biotechnology center in an attempt to develop a credential for the biotechnology POS that would 
be recognized upon graduation. 

The most-often mentioned industry-recognized credentials available from Blue and at other East 
District high schools were, in engineering, NCCER, OSHA, and PLTW; in IT, CISCO, 

CompTIA A+, Microsoft Office, C++, and Oracle; in culinary, the National Restaurant 
Association’s ServSafe certification; in health sciences, the CNA; and in automotive, NATEF. In 
nearly all cases, participants noted that due to the recession, students bore the cost of taking these 
exams. Many participants noted that although students benefited from taking the exams, many 
could not afford them. 

Case Study: NATEF Automotive Programs 

Over the course of this study, we noted a reduction in the number of automotive programs in the 
district. By study’s end, Blue’s automotive POS was shuttered, along with several other district 
automotive programs, due to their high costs and comparatively low enrollments. Students in 
these programs were offered the opportunity to switch to another program or attend another 
school. One such student — a Blue automotive student whom we met in 10th grade — was the only 
East student we interviewed who not only knew about the future certification for which she was 
preparing, but also spoke knowledgeably and enthusiastically about graduating, taking the ASE 
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qualifying exam, and using it to work toward an advanced degree in automotive engineering. In 
her senior year, after the Blue automotive program closed, this student was compelled to return 
to her zoned high school, which had no automotive program. 

A comparison site automotive teacher expressed his frustration at these closures as well as the 
pressures on him as an instructor of the demanding NATEF ASE-preparatory curriculum. As he 
noted, the curriculum was the same whether it was offered at the high school or the community 
college, with the exception of some curricular additions made by the state (e.g., career guidance, 
car maintenance) in order to ensure high school graduates’ career readiness. Program graduates 
were prepared to continue their studies at ECC, where their high school courses were accepted 
for credit, shortening their time to a two-year degree by one semester or more. At least until the 
recession, high school students could also take automotive courses at ECC. Unlike the less 
stringent program leading to NATEF’s GST credential, the ASE-preparatory curriculum features 
strict guidelines for learned tasks, skills, and standards. This intensity made it that much more 
difficult for him to reach disinterested students. Each semester, students were placed in his 
classes merely because they had empty slots in their schedule — such students usually lacked the 
motivation to complete the demanding curriculum. A different perspective was taken by the 
instructor of another comparison school automotive program. This teacher recognized that, 
although his program aimed to prepare graduates to receive the additional training they needed to 
become ASE technicians, not all students had the same level of interest. His goal was to help 
them understand that: 

There’s just so many aspects to the automotive industry that they could become interested 
in. . . I just want them to be successful and know that there’s all kinds of options. Not be 
so rigid that. . . I turn anybody that doesn’t want to be a technician away, or turn them 
off. . . I have to be flexible enough to. . . recognize some of their talents. . . even though it 
may not be automotive technician. 

Prior to the fourth year of our study, this teacher’s automotive program was shuttered. 

State and Local Legislation or Administrative Policies 
Promote POS Development and Implementation 


State Policies 

East District’s state exerts strong control over educational policy, especially CTE-related policy; 
as such, many actions taken at the state level impinged both positively and negatively on how the 
district’s POS were developed and sustained. 

State funding and other resources. This state provided ongoing funding specifically for CTE as 
well as extra funding for schools with large populations of students from low socioeconomic 
status backgrounds. Many schools in East District qualified for this extra funding. Blue’s 
principal told us that although a comprehensive high school would invariably apply such extra 
funds to its core departments, at Blue, he also used such funding to improve POS classes and 
purchase needed equipment. 
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Establish formal procedures for the design, implementation, and continuous improvement of 
POS. State-generated POS curricula, standards, and assessments were developed with input from 
secondary and postsecondary faculty members from across the state. We spoke to many East 
teachers who had participated in state curriculum committees. Industry representatives were also 
involved in the review of POS curricula, standards, and assessments. 

Require secondary students to develop an Individual Graduation or Career Plan. East’s state 
offered several options for earning a diploma, depending upon a student’s concentration in 
college preparatory courses, career preparatory courses, arts programs, ROTC, or some other 
combination of courses or programs. Based upon these available diploma options, students 
worked with guidance counselors to choose classes each year, which they tracked on four-year 
course-taking plans. Parents were required to sign off on students’ course selections. This annual 
process, as well as the four- year plans themselves, appeared to be designed to help counselors 
schedule students into courses and ensure that they earned the requisite number of credits, rather 
than to help students plan for future careers. No student we interviewed mentioned using his or 
her four-year plan at any time other than during course registration periods. Plans also did not 
record the results of any career interest inventories that students may have taken, or any other 
activities undertaken in preparation for the transition to college and careers. One counselor noted 
that although comprehensive plans that followed students from middle school through high 
school had once been in use, such plans were no longer required. The district’s CTE director said 
that the state did not mandate such highly individualized plans. 

Provide resources for long-term sustainability of POS. As part of its college and career 
readiness initiatives, the state was developing more POS and STEM programs, drawing upon a 
statewide coalition of secondary and postsecondary faculty and industry partners. Schools with 
PLTW programs worked together on grants to enrich their programs. 

This state allowed teachers to develop and test curricula, with the expectation that such curricula 
would pass through the state’s approval process. To receive state approval, a new curriculum had 
to include an item bank of test questions and be reviewed by a committee comprised of 
postsecondary faculty and at least 80% industry representatives. According to one STEM 
teacher, the state’s intensive review requirements not only served to squelch teachers’ desire to 
develop new and different curricula, but also moved so slowly that some newly developed 
programs ran the risk of being out of date by the time they were approved by the state. 

The state worked with postsecondary faculty and industry partners to develop vertically aligned 
postsecondary curricula for use statewide. State-convened curriculum improvement groups 
united faculty from every college offering a given program. Post-recession, such groups were not 
able convene frequently, although some fields, like IT, changed so frequently that the state 
needed to quickly alter its curricula to keep up. 

How to develop POS that could be sustainably and responsively updated over time presented 
challenges to both the state and the district. One district CTE administrator believed that 
conceiving of POS as prescribed, on-paper course sequences was outdated. Given the rising 
popularity of online and dual credit course-taking options, fewer students were remaining in high 
school for the full four years. In this “new world,” she believed the district needed to think more 
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“futuristically” about its POS; the future included a program consisting of six years of education 
offered in a five-year period, culminating in both high school and postsecondary credentials. 

This accelerated five-year plan had been approved by the state for pilot testing. 

Other relevant state policies. The state defined a CTE concentrator as a student who completed 
four credits in a CTE program area, the last of which was credit from a capstone course. 
However, CTE concentrators did not have to follow a specific course sequence. By contrast, 

POS completers, as defined at the district level, had to earn at least four credits in focused, 
progressively more intensive course sequences, culminating in a capstone. Students in Blue’s 
POS academies were expected to take POS-related electives. 

District Policies 

Local funding and other resources. The recession led to major cuts to teacher and administrator 
positions during the early years of the study, and even some school closures. Most of these 
closures were at the elementary and middle school levels, but high schools were also closed; 
students from closed schools were transferred to other schools. In the case of students attending 
district magnet programs (e.g., POS, IB, or arts programs), students were offered the option of 
pursuing their programs at other schools. To ease their transition, high schools receiving large 
numbers of transferred students quickly moved to implement curricula offered at the closed 
schools so that students could complete their programs. For example, Heliotrope, one of our 
comparison sites, de-emphasized its comparatively under-enrolled pre-engineering program in 
order to adopt programs in early childhood education and marketing that were needed by the 
many transfer students it received. 

The recession meant that CTE programs at all district schools faced cutbacks. Although some 
programs lost teachers, both the district and the schools themselves made efforts to “protect the 
classroom” by focusing cuts elsewhere, where possible. One position targeted for cuts was the 
school-based administrative career coordinator; some schools lost their dedicated coordinator 
and were compelled to share this position with another school. Schools with limited technology 
resources resisted the pressure to “double up” students on computers because students needed 
free access to technology in order to perfonn well on end-of-tenn technical assessments. 

The recession also affected operations in the district’s CTE department. During one especially 
difficult year, districts were required to return money to the state with little notice; orders of 
needed materials “that were in delivery en route [were] just cancelled, you know, on the truck.” 
With its budget slashed close to 40%, the district’s CTE department concentrated on its core 
missions: supporting technology, basic supplies, professional development, and CTSOs. 
Activities deemed less essential were cut, including monthly curriculum alignment committee 
meetings. Although school administrators sought ways to work around budget cuts — say, by 
filling an open administrative position with a classroom teacher — district funding formulas 
precluded many such efforts. Some participants thus complained of “downtown” bureaucratic 
procedures that required them to “go through ten levels of approval to get anything.” 

One comparison school CTE teacher was well aware that given the district’s constrained budget 
and restrictive funding formulas, it made more sense to hire one English teacher who could teach 
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multiple sections, each serving 30 to 35 students, rather than one CTE teacher serving small 
numbers of students in program-specific upper-level courses. Given such pressures, some 
expensive-to-maintain programs, like the CNA strand of health sciences, were cut. Other 
programs, according to one teacher, sought to perform well but keep a low profile: “If we’re 
quiet and do our business, and get good scores, they’ll leave us alone. But if you’re costing them 
money. . .” Overall, a number of teachers in more traditional CTE programs at the comparison 
sites believed that the recession offered a ready excuse to close their programs and re-allocate 
funds to more popular initiatives (e.g., STEM, PLTW). 

By 201 1, as the economy slowly recovered, the district was able to restore some teacher 
positions and purchase new equipment for some programs. 

Ensure opportunities for any secondary student to participate in a POS. Cuts to the district’s 
transportation budget led to a reduction in the number of bus pickups made near students’ homes 
and the creation of a more limited number of regional “shuttle stops.” For students enrolled in 
distant magnet or POS programs, this meant having to secure either private or public 
transportation to the shuttle stops. Although the district saved money through this new policy, it 
created considerable upheaval for magnet schools and students. Students who could not obtain 
transportation were compelled to give up their programs and return to their zoned high schools. 
As one teacher lamented, “Kids need consistency and structure.” 

Provide resources for the long-term sustainability of POS. East’s long-term goal was to 
implement POS in all of its comprehensive high schools and to gradually eliminate resource- 
draining singleton classes in favor of groups of related courses linked to selected Career Clusters, 
as we described in an earlier section on non-duplicative course sequences. The district’s CTE 
department sought to help principals choose programs that would help them meet broader school 
goals, weighing a program’s popularity against the need to attract and retain students from feeder 
middle schools with similar magnet programs. 

East District supported its high school CTE programs with dedicated school-based personnel, 
including career coordinators who served as point people for all CTE services and activities. 
Career coordinators ensured that a school’s CTE or POS-related instructional, material, or 
purchasing needs were met and conducted instructional visits. Instructional coordinators were 
responsible for administering assessments and reporting outcomes. During the recession, many 
schools lost one or both of these positions; in some cases, schools shared career coordinators, 
and teachers picked up some of their duties. As schools lost more CTE teaching positions, 
however, teachers could not continue to absorb these additional tasks. As such, some coordinator 
positions were brought back. 

The district supported transportation for field trips within the county. At Blue, academy teachers 
sought to give students opportunities to visit workplaces related to their POS. Given the limited 
number of biotechnology companies in the area, one of Blue’s biotechnology teachers solicited a 
donation from a previous employer that allowed her to take her students on a field trip to a 
manufacturer located in a different county. 
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Other relevant district policies . Students enrolled in any of East’s lotterized magnet programs 
who failed a required magnet course (i.e., a POS course) were given extra academic support for a 
semester. If they failed to improve by the end of that semester, students could be sent back to 
their zoned high schools. 

Ongoing Relationships Among Education, Business, and Other Community Stakeholders Are 
Central to POS Design, Implementation, and Maintenance 

Business and Industry Partnerships 

East’s CTE programs and POS were governed by “umbrella” state-level advisory boards (e.g., 
spanning Career Clusters tied to sector strategies) comprised of postsecondary and industry 
partners. Industry partners were involved in all levels of curriculum development in order to 
ensure that CTE curricula met industry standards; the state also required that industry partners 
review all end-of-tenn test questions as they were being developed. According to the district’s 
CTE director, business and industry involvement was not only “good PR,” but also a valued 
mechanism for developing a pipeline of skilled workers to replace a graying workforce. 

Although the recession forced many business partners to reduce or withdraw some of the 
material, financial, or personal assistance they provided to the state, the district, and individual 
schools, many interviewees stated that business partners continued to offer abundant support. 
Such support included donated equipment, work-based learning opportunities (coordinated 
centrally by the district), funding to support field trips and other special events, scholarship 
assistance, and mentoring and volunteer hours. Academy coordinators and career coordinators 
facilitated the process of building and maintaining relationships with these partners. 

Overall, there appeared to be little difference in the types or amount of business and industry 
support offered to Blue Academy versus our comparison sites, which offered “regular” CTE in 
comprehensive high school settings. Support also flowed in the other direction, with schools 
reaching out to businesses and the community to offer help and expertise. At Blue, for example, 
e-Commerce students worked to create and update websites for small businesses; they also 
offered marketing plans to businesses featured in the local business newspaper. At Indigo, 
students in the school’s PLTW architectural design program joined with students from other East 
District ACE (Architecture, Construction, and Engineering) Mentor programs 5 to redesign a 
derelict shopping mall into a vibrant new mixed-use facility. 

Each school also had school-level advisory boards; at Blue, NAF required one board for each of 
its three academies. At Emerald, participants interviewed in the third year of the study described 
an upcoming open house event at which they planned to showcase the school’s emerging NAF 
Academy of Engineering to dozens of business and community partners, including nearly three 
dozen academy advisory board members. These members were described as providing input on 
the academy’s curriculum and work-based learning experiences. The school saw these partners 
as providing a valuable supplement to the otherwise state-directed CTE curriculum. Emerald was 


5 A national initiative with affiliates in nearly every state, ACE Mentor programs bring schools, students, and 
industry professionals together for a year’s worth of concentrated lessons, teamwork, career exploration, and a 
hands-on construction project. We found ACE programs in East and South Districts. 
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also remarkable for its foundation: Funded by generous alumni, the foundation paid for 
professional development and personal enrichment for faculty and students. Every few years, the 
school held pledge luncheons, catered by the school’s award-winning culinary students, to 
generate additional donations. Emerald’s principal and career center coordinator agreed that 
finding business and community partners to support the school’s programs was never a problem. 
As the coordinator stated, “We have a warmth in the community and a willingness to jump in.” 

Student experiences. Although not mandated by any district CTE program or POS, work-based 
learning experiences in the form of internships were available to students who wanted them. 

CTE teachers also arranged interactions with business and industry. In one Blue IT class, 
students worked on a long-term project that involved building a virtual computer system and 
creating a sales pitch to describe it. The teacher arranged for these presentations to be filmed and 
reviewed by a Microsoft representative who returned feedback on their technical and marketing 
know-how. 

Community Partnerships 

As in our other two districts, East schools not only received valuable support from local service 
organizations, non-profits, and charities, they also sought to give back to these organizations. 
Indigo’s high-energy principal described doing “something every week” with “homeowners 
associations, ... feeder schools, [and]... various civic groups in the community, whether it’s the 
Y or Kiwanis, Rotary, Optimists, the Masons. . .and on and on and on. . . I’m at a lot of sports 
events. We participate in any community activity we can.” This principal also described how 
highly involved his school’s CTSOs — including DECA, FBLA, and FFA — were in giving back 
to the community. 

Postsecondary Partnerships 

Because articulation agreements and curriculum alignment were largely negotiated at the state 
level, there was much less incentive in East District for teachers to seek out or nurture personal 
relationships with postsecondary partners. However, participants at every site commented on the 
positive, supportive relationships they enjoyed with the region’s highly regarded community 
college, state universities, centrally located Historically Black College and University (HBCU), 
and local trade-technical and culinary schools. Postsecondary institutions, especially ECC, sent 
representatives to the schools to make presentations and assist students with applications, testing, 
financial aid, and scholarship infonnation; sponsored campus visits for students and parents; 
hosted informational meetings for high school guidance counselors; and in one special initiative, 
sent university students to the district’s engineering and IT classrooms to give hands-on 
presentations about the latest technologies. 

The local community college. ECC administrators, faculty, and counselors all described efforts 
to create seamless pathways between the district’s programs and its own, thus increasing 
enrollments. As our data collection efforts in East drew to a close, ECC was in the process of 
working with the state and district to coordinate a new dual credit program that would offer more 
prescribed, streamlined pathways to earning college credit. Overall, however, over the course of 
our study, district support for credit transfer opportunities drastically declined. A highlight of our 
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first site visit to East was an early college enrollment program that allowed students to take 
courses on the college campus with district support for books, tuition, and transportation. This 
program had disappeared by the last year of our study, having been largely replaced by cheaper 
online coursework. 

ECC remained a popular and affordably priced option for many East District graduates. The first 
principal we interviewed at Blue commented that as many as 80% of his graduates went on to 
enroll there. However, he and others at Blue also strongly encouraged students to apply to four- 
year universities. Some ECC faculty who had worked closely with Blue to align their programs 
and accommodate high school students’ schedules said that many of the students who earned 
credits from their program ended up attending four- year universities on scholarships instead. 
These faculty members questioned the value of their investment in Blue’s programs. 

ECC also cultivated relationships with the district and the state university system, as in the case 
of a college transfer success class that was being piloted with a group of district students who 
had been denied admission to the state university because of their low academic performance. If 
these students agreed to attend ECC and successfully completed two semesters of intensive 
remediation and university-preparatory coursework, they would be guaranteed admission to the 
state university. 

The state university system. When interviewed in 2010, the district’s CTE director described 
efforts to align the district’s engineering programs — particularly its PLTW programs and NAF 
engineering academies — with one of the state universities as still “in its infancy.” He believed 
that state university officials realized how few district students were enrolling at the college 
compared to students from out of state, and recognized that they could do a better job of 
marketing and recruiting from within the district’s high-quality programs: “I think they’re 
viewing [CTE] in a different light. . . They’re seeing the coursework that we’re providing and 
they didn’t know it could lead directly into the university system, and they didn’t know the 
percentage of students who were already college prep ... in the district.” 

Sustained, Intensive, and Focused Professional Development Opportunities for 
Administrators, Teachers, and Faculty Foster 
POS Design, Implementation, and Maintenance 

East District provided professional development to all school personnel on a wide array of topics 
related to assessment and accountability, technology use, college and career readiness, state 
graduation requirements, enriched academics, CTE curricula, POS, school safety, pedagogical 
strategies and classroom management techniques (e.g., differentiated instruction, literacy 
interventions, professional learning communities, block scheduling), and counseling issues. The 
district also provided support for new teachers entering the profession from business and 
industry. Study participants described professional development as occurring during summer 
sessions, Saturday workshops, and regularly scheduled in-services throughout the school year. 
Professional development was available onsite, online, at national conferences or trainings, and 
at local and regional postsecondary institutions. 
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Across all four years of our study, many participants reported that a great deal of professional 
development addressed the district’s electronic assessment and accountability systems, which 
included online test item banks, data reporting and analysis tools, and electronic student 
education plans (e.g., four-year graduation plans). Administrators, CTE instructional 
coordinators, career coordinators, academy coordinators, guidance counselors, and teachers all 
mentioned participating in trainings related to these tools and systems. Most agreed that the 
district’s goal was to help schools use data to improve instruction and increase student 
achievement. As one principal stated, the district was “trying to minimize the amount of data 
they’re giving us, but help us understand how to use that data.” Training related to these systems 
was most often described as helping teachers understand how to upload items into and use test 
item banks, analyze system-generated data reports, use those reports to identify deficiencies, and 
work together to enhance their teaching strategies. A number of interviewees noted that CTE 
teachers had an advantage in adapting to these new assessment systems because they had been 
using state- and district-sanctioned CTE test item ha nk s and data tools for many years. Core 
academic teachers, by contrast, were familiar only with the state’s high-stakes tests and had 
never before been asked to upload their classroom assessments to a centralized system. Despite 
CTE teachers’ perceived advantages in using this system, two coordinators agreed that the 
pressure to use these assessment systems could be “overwhelming even for seasoned teachers,” 
to say nothing of new, alternatively certified teachers. 

Given East’s heavy investment in instructional technology, many interviewees noted that they 
had participated in technology-related professional development, and most said that they were 
fully supported by their principals and the district in their efforts to incorporate technology in 
their classrooms. Frequently mentioned were trainings related to Moodle, the district’s online 
teacher community of practice, document-sharing hub, and virtual classroom environment; 
instant student response systems (i.e., “clickers”); common classroom equipment (e.g., graphing 
calculators, document cameras, projection systems); and various instructional software packages. 

General District CTE Professional Development 

East’s CTE department provided professional development to all schools offering CTE, whether 
that CTE was offered in the fonn of stand-alone classes or programs or fully fledged POS. 
Participants described these trainings as being delivered by district CTE personnel, school-based 
CTE instructional coordinators, academy coordinators, personnel from external accrediting 
agencies (e.g., NAF, PLTW, NATEF), the local community college or universities, and 
somewhat less frequently, industry partners. 

During the course of our study, the district’s CTE department engaged in a major effort to 
explain the structure of the state’s sector strategy- aligned career pathways and how these 
connected to course sequences and graduation requirements within CTE programs and POS. 
District-trained instructional coordinators and academy coordinators were primarily responsible 
for helping guidance counselors and CTE teachers understand these requirements so they could 
communicate them effectively to students and parents. 

As noted, a major district initiative during our study was the implementation of the PLTW 
curriculum in nearly all district high schools. The district’s CTE director explained that although 


112 



it required substantial investments in intensive teacher training (sometimes out-of-state), 
materials, and software, PLTW offered the best value in tenns of academic rigor, curricular 
quality, and improved outcomes for all students, not just those enrolled in PLTW, by creating a 
culture of high expectations in each school. By embracing models like PLTW, the district’s 
philosophy was that it had to “raise the bar for every kid in your school and . . . move forward 
from there. That’s what we’ve done here rather than. . .[seek] the quick fix.” 

At Emerald, for example, teachers, administrators, guidance counselors, and instructional 
coordinators were committed to building a NAF-accredited Academy of Engineering featuring 
the PLTW engineering curriculum. With financial support from the school’s foundation, the 
district, and industry partners, participants reported engaging in both PLTW and NAF trainings. 
Having started with one cohort of students and a limited number of courses, the school was 
gradually adding courses and grade levels to its curriculum. Data collection and analysis were 
identified as very important to this process, as the school was keen to earn distinguished status 
for its NAF academy. 

Abundant occupationally-specific professional development was available from national 
certifying bodies and regional postsecondary and industry partners. An Indigo automotive 
teacher described how he participated in 20 hours of NATEF training each year and also 
received training on changes in automotive technology from General Motors, Chrysler, and 
ECC. Similarly, a business academy teacher at Blue noted that she had received continuing 
education support from her program’s business partners, including training in how to use 
computer programming and data analysis software packages like SAS. This training allowed her 
to develop engaging, hands-on projects for her students. Some highly-skilled teachers also 
trained their fellow teachers. 

Professional Development for Guidance Counselors 

Guidance counselors noted that although most school-based professional development was 
focused on teachers, they were otherwise well-supported with the counseling-specific training 
needed to maintain their licensures. Trainings were generally reported as focusing on scheduling, 
testing, college preparation, student safety and mental health issues, and to a lesser extent, career 
preparation. When asked about POS, guidance counselors at all study schools reported that they 
had received some degree of professional development from their instructional coordinators, 
career coordinators, or academy coordinators related to the state’s changing graduation 
requirements, POS, and course sequences. Counselors also noted that both the district and ECC 
provided them with abundant information related to credit transfer opportunities, admissions 
tests, financial aid, scholarships, and enrollment processes. All counselors mentioned that ECC 
and state and regional universities sent representatives to school college fairs and facilitated 
campus visits. 

Professional Development Provided by the Community College 

ECC was described by many participants as a major provider of professional development and 
credentialing for district teachers in both academic and occupational programs. Faculty in one of 
the college’s engineering-related programs described how they helped credential high school 
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teachers to offer college-approved dual credit courses. These faculty members were intimately 
involved in the development of aligned, sequenced POS between the district and the college and 
were strongly motivated to protect the integrity of their programs. As one college faculty 
member stated, “[our] accreditations rely on a certain level of expertise of our faculty. That’s one 
problem in high schools — there’s not many engineers teaching in high schools.” 

ECC admissions counselors noted that they spent a great deal of time working directly with 
students, parents, counselors, and teachers at district high schools so that they understood college 
enrollment and admissions policies, credit transfer options, and the full range of financial aid and 
program opportunities available. ECC hosted events both in the schools and on its own campus. 

Systems and Strategies to Gather Quantitative and Qualitative Data on Both POS Components 
and Student Outcomes Are Crucial for Ongoing Efforts to Develop and Implement POS 

State Assessments 

Little to no distinction existed in this state between state assessments and other academic 
assessments: All were state-run and housed online. The same held true for CTE technical skills 
assessments, described later in this report. In lieu of offering high-stakes math, English, and 
science tests in the 10th grade, all assessments in this state were end-of-term assessments 
administered at the end of every core course. In order to graduate, students had to pass the 
Algebra I, English, and Biology end-of-tenn exams; these courses were often but not always 
taken in the 10th grade. 

This state adopted a web-based assessment system as its statewide growth model that allowed 
users to track students’ longitudinal perfonnance data, including their grades and performance on 
interim and end-of-term assessments, for the length of their residency in the district. Given past 
perfonnance data, the system could make individual student-level perfonnance projections for a 
given course and identify which students were at risk of failure, thus allowing teachers to target 
those students with instructional interventions. The system also measured teachers’ effectiveness 
with high-achieving, mid-level, and low-achieving students. According to one principal, teacher 
perfonnance data were used to protect, not punish, struggling teachers by helping them identify 
areas of instructional weakness and, supported by peer mentoring or targeted in-services, adopt 
new practices to better reach all students. 

At least as initially implemented, unlike the CTE assessment system, academic assessment 
results were not broken down at the individual student or question level; rather, the system 
offered trend data — student growth and teacher growth over time. However, academic and CTE 
assessments were similar in drawing upon state-provided test item banks that allowed the state to 
track achievement on the same items statewide. As noted elsewhere, CTE teachers were already 
familiar with the use of these test item banks; academic teachers were not. Further, when the 
state implemented an interim assessment system, CTE programs were able to quickly populate 
their interim assessment test item banks; no extra support was available to help pay for the extra 
time academic teachers needed to populate their interim assessment test item banks. 
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Principals reported that assessment results varied based on the availability and quality of 
teaching staff. Prior to the hiring of the first principal we met at Blue Academy, the school had a 
shortage of certified math teachers and instead employed long-tenn substitutes with limited 
classroom management skills. Low math test scores reflected these teachers’ inexperience. Three 
years later, after the new principal stabilized hiring and achieved consistency in the math 
department, scores rose. Similarly, at Neon, non-certified science teachers produced low science 
scores. Once the school hired a certified science teacher, scores rose. 

Principals used other strategies to improve students’ test scores, including online remediation. At 
Indigo, 100 ninth graders who were reading below grade level were given a remedial reading 
program and online coursework supported by some of the school’s best teachers and facilitators. 
Students then took the English I end-of-tenn assessment online; 92 of the 100 passed both the 
course and the exam. According to Indigo’s principal, the experience “allowed us to meet kids 
and develop skill sets depending on where they were.” He stressed that although online courses 
could be a great economic decision, they were not necessarily a great educational decision if 
resources were not devoted to their successful implementation. 

Other Assessments 

The state board of education mandated that all juniors take the PS AT and all seniors take either 
the SAT, ACT, or a WorkKeys technical assessment. Students in the federal free or reduced- 
price lunch program could take the SAT or ACT at no charge. 

As part of its outreach efforts, ECC staff visited all East high schools to encourage students to 
participate in its early registration program. During these visits, students took ECC’s placement 
exam. If their scores were good, students could keep that administration of the exam; if their 
scores were not good, students had time to improve and retake the exam. 

Content Standards Define What Students Are Expected to Know and Be Able to Do to 
Enter and Advance in College and/or Their Careers 

Participants’ definitions and perceptions of college and career readiness varied across East 
District. College preparation — and a general expectation that all students would strive to attend 
college — was the most commonly expressed goal. As one Blue pre-engineering teacher bluntly 
stated: “All of these students are college bound. . . We have the mentality that these students are 
going to college and we’re preparing them in that fashion.” 

In addition to urging students to prepare for college and earn college credits, however, teachers 
and administrators also stressed preparing for careers. As one medical careers teacher put it: “I 
really want to see all my kids in some sort of career that I can look back and say, ‘I remember 
her. Now she’s Dr. Such-and-Such,’ or ‘She’s head over research at this university,’ or 
something like that. So we talk on a daily basis about their future because I have mainly ninth 
graders.” 

Another avenue for career preparation was through work-based learning and opportunities to 
earn industry-recognized credentials and certificates that would have value in the workplace. At 
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Blue, portfolio preparation was a cross-academy initiative designed to combine POS research 
and college application planning. Students kept career journals in which they explored different 
jobs linked to the medical fields and concepts they were studying. They also compiled 
transcripts, recommendations, SAT scores, community service hours, evidence of extracurricular 
activities, and other materials that they could readily present to college admissions officers. A 
computer science teacher described how students with carefully prepared portfolios had been 
“accepted on the spot” by admissions officers attending an on-campus college fair. School-wide, 
teachers and administrators sought to create a culture in which students “look prepared. . . [like] a 
student that’s ready to be a freshman in college.” 

Definitions of college and career readiness varied at the four comprehensive high schools in our 
comparison group. Emerald’s principal and career center director both emphasized the role of 
work-based learning, partnerships with local businesses, job readiness workshops, and career 
guidance and exploration in creating smooth transitions into either college, jobs, or military 
service. At Indigo, the principal believed that his students needed to graduate with coursework, 
college credits (both general education and CTE), and skill sets that would dovetail with their 
career choices and allow them to pursue any form of postsecondary education, whether that was 
at a two-year or four-year college or a technical school. 

At Heliotrope, a self-described college preparatory school, interviewees cited the school’s 
expectation that all students would go to college and frequently referred to college fairs, college 
advising, scholarships, and the school’s partnership with ECC. However, the career coordinator 
also described her efforts across programs to better educate students about the district’s Career 
Clusters approach and to help “build [their] transferable skills to be able to go into any type of 
career to be successful.” 

Interviewees at Neon, East’s arts school, saw themselves as readying students to graduate with 
the same or better preparation for arts-oriented careers as college students. Such preparation 
included college and career advising tailored to students’ arts focus. As the school’s drama 
department chair explained, students were counseled to understand the differences between 
regular colleges and specialized conservatories; the latter, she noted, did not always offer credits 
that could be transferred to other institutions. Students were also extensively advised on their 
college application and audition materials. Although less fonnal CTE was offered at Neon 
compared to the district’s other high schools, its career coordinator perceived college and career 
readiness as offering rigorous academics coupled with work-based learning that would enable 
students to make empowered college and career choices after high school. 

Guidance Counseling and Academic Advisement Help Students to 
Make Informed Decisions About Which POS to Pursue 

In East District, state graduation requirements for our cohort mandated an academically rigorous 
college preparatory curriculum for all students. Students enrolled in the district’s POS (as at 
Blue) or in CTE programs at the comprehensive high schools could earn a career-ready diploma 
provided they completed a series of sequenced, non-duplicative courses. As with the other two 
districts participating in this study, East did not necessarily refer to these sequences as “programs 
of study.” As previously described in the non-duplicative sequences section of this report, district 
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personnel perceived their approach as broader and more flexible, giving students the option to 
complete their sequences in as little as three years. All CTE curricula included a state -mandated 
career guidance component delivered by CTE teachers through classroom instruction and career 
exploratory activities. 

At ECC, the admissions counselors we interviewed supported the district’s approach, believing 
that students who followed a POS were more directed, better advised, had better grades, and 
were generally better students. The college helped ease students’ paths from high school to 
postsecondary by bringing high school guidance counselors to its campus to educate them about 
its programs and sending admissions counselors to the high schools to provide students and 
parents with information and early registration support. 

Guidance Counselors ’ Roles and Responsibilities 

As in the other two districts participating in the study, East guidance counselors reported 
spending much of their time engaged in scheduling, course registration, college planning, and 
administrative tasks. At Blue, for the first three years of the study, guidance counselors were 
academy-specific and worked with the same group of students across all four years. This allowed 
counselors to not only develop deep knowledge of their academy’s POS, but also personal 
histories with the students in them. We perceived this system as a school strength. However, 
prior to the fourth year of our study, the loss of a staff member and the recession led Blue to 
revert to alphabetic student caseloads. This model prevailed at the district’s comprehensive high 
schools, including at our four comparison schools. 

One Blue guidance counselor explained how during pre-registration periods, she reviewed 
students’ transcripts and preselected classes for them based on their POS; during scheduled 
counseling sessions, students and parents could select different electives or customize students’ 
course selections based on their college and career goals. She conducted credit checks twice 
yearly, and the district also mandated on-track-to-graduation credit checks in senior year. 

Other Support Personnel 

At all district schools offering CTE, CTE teachers and programs were supported by career 
coordinators who delivered career exploration activities, coordinated internships, implemented 
technical skills assessments, and facilitated student participation in dual enrollment, online 
coursework, and credit recovery programs. Trained by the district’s CTE office, career 
coordinators also educated guidance counselors and teachers about district CTE initiatives, 
including POS, technology adoption, and graduation requirements. At Emerald, an instructional 
coordinator worked with teachers to integrate technology into their curricula. Emerald also had a 
career center staffed by a full-time director. Other district schools shared a career coordinator 
with a neighboring school. At Blue, academy coordinators — de facto department chairs — 
undertook the tasks performed by career coordinators and served as liaisons between the 
administration and their academy’s teaching and counseling staff. 
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Classroom-Based Career Guidance 


Although all of Blue’s academies embedded career guidance and career development into their 
curricula, its Medical and Biotechnology Academy yielded such striking examples that we offer 
it as a case study. In a senior-level Biomedical Engineering class that we observed, an alumna — 
now a lower classman at a prestigious in-state university — came to spend the day with her 
former classmates and share her thoughts regarding the realities of college. This young woman 
talked about how well Blue students were being prepared for the rigors of college, explaining 
that many of her fellow university students lacked her knowledge and lab skills. Such visits were 
a common occurrence, according to the teacher. This particular class featured a number of 
noteworthy examples of career guidance and skill development, including a reference to a 
previous module in which the class built and maintained online stock portfolios in an effort to 
understand biotechnology startups and the obstacles they faced finding funding and turning a 
profit. Although the main focus of the class was practicing DNA analysis techniques, this 
reference highlighted how the teacher incorporated the business aspect of biotechnology to her 
curriculum. 

Professionalism was a major component of this academy’s curriculum. Students developed 
portfolios in which they maintained resumes, work samples, community service hours, action 
plans, recommendation lehers, and transcripts that they could use for job or college applications. 
Students also kept journals in which they explored careers linked to each unit they studied (e.g., 
if learning about heart disease, a student might prepare an entry on cardiologists). As the 
academy coordinator noted, these journals exposed students to a wide range of career options 
within a POS. The goal was for students to understand that career trajectories were not linear, 
and that different levels of education and training provided access to different kinds of jobs. 

Medical and Biotechnology Academy students also had many opportunities to engage in work- 
based learning, CTSO-sponsored activities, and other forms of career exploration. One senior 
said that she interned at a local hospital during the previous summer and also spent time 
shadowing an emergency room nurse. She actively volunteered at four health care-related 
community organizations and, as a HOSA member, participated in school blood drives. The 
academy invited health professionals to speak to students and sponsored visits to universities 
with well-regarded medical programs and field trips to health-related companies. The 
biotechnology teacher noted that one of these field trips had led to an excellent job for a student. 

Career guidance and career exploratory activities were also a part of the curricula at our 
comparison sites, but in a less concentrated and sometimes less well-resourced way. At Neon, 
the district’s arts school, career guidance was arts-inflected. For example, a senior-level theater 
class featured advice and feedback regarding how to prepare for college auditions. The teacher 
spoke at length about how the program sought to professionalize students, working with them on 
how to dress, obtain head shots, build resumes, conduct mock auditions, and choose agents and 
managers. Program faculty wanted their students to graduate as well prepared, if not more so, 
than most college students. The program also sought to arrange internships at local theater 
companies at which students could perfonn theater tech and stagehand work, but scheduling 
issues sometimes made such internships difficult to secure. Overall, work-based learning 
opportunities were challenging to organize and not overly abundant for Neon students; two 
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interviewed students suggested that they sought out internships by requesting the support of a 
CTE teacher, the CTE chair, and the school’s career coordinator. Career coordinators at other 
comparison sites — including Heliotrope and Indigo — agreed that arranging internships was 
challenging not just because of scheduling issues, but also because the recession had reduced the 
ranks of their business and community partners. An advanced business studies teacher at 
Heliotrope drew on her own personal business connections to place her students in internships. 

Innovative and Creative Instructional Approaches Enable Teachers to 
Integrate Academic and Technical Instruction and Students to 
Apply Academic and Technical Learning in Their POS Coursework 

Interdisciplinary Teaching Teams 

Blue Academy. At Blue, curriculum integration and teacher collaboration occurred during 
teachers’ planning periods; no additional time was allotted for this activity. In general, core 
academic teachers shared items that POS teachers could incorporate into their classes to help 
students meet academic standards. The POS teachers we interviewed said that they largely 
integrated their curricula themselves. 

Although limited in scope by scheduling constraints, interdisciplinary collaborations were 
appreciated by students and relished by teachers. A medical POS teacher reported working with 
science colleagues on curricular units like DNA, sharing worksheets and activities across 
departments; her students were excited about encountering the same topic in different classes. An 
engineering teacher said he found himself providing less tutoring in math after he mentored the 
school’s three new math teachers in how to integrate their instruction with architectural 
engineering problems. One of these math teachers commented that his engineering academy 
students were highly engaged by such lessons. 

Blue’s biotechnology teacher found many opportunities to collaborate across disciplines, despite 
the limited planning time available to her. Having first solicited the principal’s approval, she co- 
taught a microbiology class with a science teacher; the science teacher served as the lead teacher, 
and she led the class’s lab component. The biotechnology teacher also sought to align her 
program with the school’s earth and environmental science classes; if the school could find the 
resources to construct a greenhouse, she envisioned her biotechnology students genetically 
manipulating the plants that Blue’s earth and environmental science class students would grow in 
the greenhouse. This teacher also believed that the school should foster data sharing across, not 
just within, academic and technical departments; she asked the principal to allow POS and 
academic departments (e.g., biotechnology and science, or computer programming and math) to 
work together to analyze their student performance data, identify areas of curricular weakness, 
and spend the summer developing lesson plans and other activities to address those weaknesses. 
This teacher strongly believed that the more each department understood the other’s curriculum, 
the better they could sequence their instruction and target individual student learning needs. 

We observed only one example of interdisciplinary team teaching at Blue: a math teacher and an 
engineering teacher co-teaching a pilot PLTW course. In this example, however, the math 
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teacher remained largely silent. Team teaching appeared to be a relatively new phenomenon at 
Blue, with teachers’ roles still unclear. 

Comparison schools . Curriculum integration and interdisciplinary teacher collaboration were 
less evident at the comprehensive high schools we visited. One comparison site CTE 
administrator said that despite a national push to align academics and CTE, this state “never did 
buy into it wholly, as in, ‘Let’s sit down at the table with the academic teachers to integrate what 
we do.’ If it’s a drafting teacher and students need geometry skills, he teaches those skills, but 
it’s not aligned in any way with what that geometry teacher would be doing.” 

Overall, CTE teachers were responsible for teaching the academic skills embedded in their CTE 
curricula, not the other way around. These embedded academic skills were an implied 
component of the state’s technical skills assessments. For example, although there were no 
explicit geometry questions in a drafting assessment, students had to know geometry concepts in 
order to answer questions about drafting. 

Statewide, as academic testing expanded and became more high stakes, academic teachers 
became even less available to help other teachers integrate academics into their curricula. This 
affected all disciplines, not just CTE. As a theater teacher at Neon, the arts school, put it, “I feel 
that everybody is so stressed about teaching what they’re going to be tested on in their own 
curriculum, it’s really hard for them to move beyond that sometimes.” Further, given that 
common planning time in most schools was dedicated to single departments, there was little time 
for academic teachers to collaborate across disciplines. 

We observed some examples of curriculum integration in the comparison high schools, but most 
involved no teacher teaming. A math teacher at Indigo used examples from science and medicine 
to teach the concept of volume in her Algebra II class; this led to a discussion of professions in 
which calculating volume would be an essential job function. Heliotrope’s advanced business 
teacher heavily infused her class with reading, writing, and communication skills. Overall, the 
teachers we spoke with saw curriculum integration as a means of capitalizing on student interest 
and teaching academic skills almost by stealth. As Indigo’s automotive teacher said: 

The perception has always been that automotive, construction, horticulture. . . is not 
academic, but it is. My kids come to Indigo, [and] some of them stay the whole four 
years, and what motivates them to come is my class. Or if they’re interested in 
horticulture, then what motivates them to come is horticulture. . . It’s not that we don’t 
teach academics, because we do. We just don’t advertise.... [Students] get [academics] 
pushed down their throats, and you have to get in the back door with them. I mean, yes, 
we have to do it, and we do it. But the main reason we do it is so the car will run. 

During our study, Emerald was developing a NAF Academy of Engineering featuring the PLTW 
engineering curriculum. As part of the process, an interdisciplinary team of academic and 
technical teachers met weekly to collaborate, frequently observed each other teach, and were 
scheduled to receive training on curriculum integration over the summer. We observed one of 
their weekly meetings, at which a math teacher commented that he signed on to the academy 
because of the district’s adoption of the Common Core State Standards. He believed that the 
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PLTW curriculum not only met the standards, but was also “fun and interactive” and helped 
students “to take what they know and apply it to a real world situation.” The teachers we met 
were enthusiastic about working, learning, and teaching together; their palpable excitement 
highlighted the comparative dearth of such opportunities elsewhere in the district. Overall, we 
found that schools were not averse to teacher collaboration or integrated curricula; rather, budget 
and scheduling constraints made it difficult to change institutional cultures to support such 
collaboration. By contrast, the launch of Emerald’s NAF academy involved significant schedule 
changes; an assistant principal for scheduling was a key member of its planning team. 

District perspective. Despite the difficulties involved, district personnel recognized the 
importance of implementing common planning time and changing school cultures to foster 
curriculum integration and cross-disciplinary teacher collaboration. They also acknowledged the 
limits of their ability to effect such changes beyond encouraging principals to recognize and 
communicate the value of such activities. 

Contextualized Learning Approaches: Work-Based Learning 

Budget and transportation issues limited the extent to which East students could participate in 
work-based learning opportunities, particularly off-campus; despite these issues, however, work- 
based learning in various fonns remained popular and well-supported by district and school 
administrators, teachers, parents, and students. Many study participants described internships and 
participation in school-based enterprises or other on-campus hands-on learning experiences as an 
excellent way for students to explore careers, learn valuable skills, and prepare for both college 
and the job market. As Emerald’s principal noted: “It’s a great way to get them into an area. . . 
[and] not all kids are college-bound. However, if they get into an environment that will teach 
them the skills to pay for their college, too, that’s another benefit for the . . . internship program.” 

Work-based learning was facilitated by the state’s recently revised graduation requirements, 
which freed space in students’ schedules for additional course-taking or off-campus work. At the 
district level, three district coordinators were responsible for maintaining relationships with area 
employers, creating protocols for employer and student conduct, and monitoring internships. 
Although internships associated with particular programs that required them — allied health, for 
example — often took place as group outings, students could also request and identify their own 
internship sites, provided they secured their own transportation. Regarding group internships, 
students in a medical pathway might travel as a group to an area hospital to visit various 
departments, supervised by their teacher and identified hospital personnel. Students who 
completed internship requirements, including completing assignments and making journal entries 
regarding their experiences in a web-based system, received a half-credit on their transcripts. 

Work-based learning took many forms, including school-based enterprises, participation in 
business and community-sponsored in-school activities, and internships and job shadowing 
experiences at area businesses. Students in the culinary pathway at Emerald were required to 
work in all capacities in the school’s on-site restaurant, bakery, and catering business and to 
complete mandatory after-school service hours at school functions. Many culinary students also 
sought and received external job placements in community restaurants. At Indigo, students in the 
school’s design and drafting program were invited to participate in a competition to design a new 
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mixed-use facility at the site of an abandoned mall. Neon’s career coordinator noted that 
although it was sometimes a challenge to educate his arts-focused students regarding the 
availability of internships and CTE credits for their work, he had several students currently in the 
internship pipeline, including one student whose internship with an area fashion designer had 
turned into steady employment for over a year. During one site visit, advanced business studies 
students at Heliotrope were preparing portfolios related to their career interests that the teacher 
was using to identify internship sites, which included a community organization focused on 
developing entrepreneurs, a photography studio, and a clothing store for a student interested in 
fashion merchandising. In Blue’s IT program, work-based learning took the fonn of community 
service: Students repaired discarded laptops for donation to community organizations and also 
built their own towers using donated parts. 

Contextualized Learning Approaches: Project-Based Learning 

Project-based learning did not appear to be a district-wide, cross-curricular mandate; however, 
all East seniors undertook self-selected long-tenn graduation projects related to their studies and 
documented those projects in a portfolio or binder. Although formally overseen by English 
teachers, graduation projects were often related to students’ career interests. This was especially 
the case at Blue and in high schools with career-related magnets or CTE programs. 

As a pedagogical strategy, project-based learning was most widely and consistently supported 
and implemented at Blue, where the school’s academy structure and block scheduling made it a 
natural fit across CTE and core academic classes, although we also observed examples of 
project-based learning at our comparison sites. At Blue, an IT teacher described how her students 
partnered with a professional sports team to perfonn analyses of game data; area businesses also 
donated equipment that allowed her students to design and build computers. In the school’s 
PLTW biomedical engineering POS, students explored genetic differences in DNA labs to 
isolate strawberry DNA and solved mysteries in long-term forensics units in which they engaged 
in blood- typing, fingerprinting, and crime scene analyses. During one site visit to Emerald, we 
observed a common planning period for teachers in the school’s developing NAF engineering 
academy. These teachers were engaged in planning a multi-phase, multi-year green roof design 
and building project with curriculum connections to science and mathematics. At Neon, project- 
based learning generally assumed an arts focus; we saw students staging theater productions, 
planning jewelry and fashion shows, and generating small business plans. 

Teaching Soft Skills at Blue Academy 

Soft skills — also described as workplace readiness skills or 21st-Century skills — were infused in 
Blue’s POS curricula. Blue’s medical POS employed patient care scenarios that allowed students 
to role-play the kinds of skills they would need in the workplace (e.g., handling a patient who 
refuses his or her medication). Medical teachers also brought guest speakers into their 
classrooms, including professors from local colleges and universities who talked to students 
about medical careers and the importance of time management in the medical profession. 
According to one teacher, the latter discussion represented a real learning experience for some 
students. This teacher also impressed upon students the connection between their behavior in the 
classroom or the workplace and future references or recommendation letters. 
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Blue’s PLTW engineering program was tied to several after-school clubs that offered students 
hands-on learning and mentoring experiences. Some of these activities specifically targeted girls 
and minority students; one such example was a district-wide “Girls in Engineering” event 
featuring engineering challenges, a business etiquette workshop, and exhibits set up by local 
college and university engineering departments. 

Team-building. Over the course of our classroom observations (13 POS, 3 academic) at Blue, 
we observed a range of activities involving student teamwork. In an English class, groups 
competed with each other to look up and use verbs in written sentences. We saw six examples of 
teamwork in POS classes, including students negotiating correct answers in a test-preparatory 
Jeopardy game, assembling a computer network cable, and dusting and lifting fingerprints. In a 
few of these observations, off-task behaviors went unaddressed; however, most students were 
highly engaged and cooperative. Senior medical academy classes offered some of the best 
examples of student teamwork: In their labs, students collegially conferred with and taught each 
other. Good teamwork and appropriate classroom behaviors were taught in this POS. In one 
lOth-grade medical science class, the teacher tolerated few disruptions: When students grew loud 
during their Jeopardy game, she announced that they would go back to using study guides “if 
you can’t handle it.” 

Critical thinking and problem-solving skills. Blue’s drafting classes offered many opportunities 
to hone critical thinking skills. During one of our observations, students were challenged to think 
in three dimensions as they imagined how a complicated object could be projected in space from 
several different angles. Students had to detennine whether an object contained hidden angles or 
shapes, then used their drafting skills to represent that object in just two dimensions on paper. 

Teaching soft skills through CTSOs. Blue’s biotechnology teacher said that she taught soft 
skills through lab write-ups and CTSO activities like leadership seminars. CTSO competitions 
related to biotechnology and the forensic sciences were team-based: Students’ combined 
individual scores on knowledge tests comprised a first round of competition; in the second 
round, teams competed to complete lab work or crime scene evaluations. Blue students placed 
nationally in these competitions. Students enjoyed such CTSO activities because they allowed 
them to socialize with students with similar interests from across the state and the nation. The 
biotechnology teacher valued CTSOs because of their power to engage students in learning and 
practicing their skills after school. When struggling students came to her for help, she 
encouraged them to participate in CTSO meetings, at which she could pair them with stronger 
students. Although CTSOs offered excellent extracurricular learning opportunities, they were not 
free: One student we interviewed wanted to join her academy’s CTSO, but knew her mother’s 
strained finances could not bear the cost. 

Teaching Soft Skills at the Comparison High Schools 

Heliotrope’s advanced business studies teacher included many skill-building activities in her 
fast-paced 75-minute class. In preparation for the SAT, students contributed words to a 
classroom word wall and completed a vocabulary-driven “ticket out the door” at the end of class. 
Students reflected on and discussed an inspirational quote from Apple founder Steve Jobs; took 
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notes on a short presentation on creating outlines; and spent time working independently on 
research reports. Two students delivered presentations on business etiquette in different foreign 
countries; at the conclusion of their presentations, their fellow students filled out feedback forms 
rating their perfonnance. Selections from these forms read out by teacher showed students to be 
supportive and professional in their compliments and suggestions for improvement. 

PLTW, offered at high schools district-wide, offered many opportunities to teach both hard and 
soft skills. At Heliotrope, an assistant principal commented on how the program increased 
students’ critical thinking and problem-solving skills by requiring them to “think and process.” 
Our observations showed PLTW being implemented with varying degrees of skills and intensity; 
although one teacher offered largely rote instruction through a PowerPoint and handouts, other 
teachers mixed their instruction with other curricular materials and facilitated student 
collaboration and problem-solving in lively group projects, like a marble-sorter construction 
activity we observed at Emerald. During this activity, which was sometimes uproarious, we saw 
students explaining to their classmates the connection between physics principles and the 
machines they were assembling. 

Emerald’s principal said his school tried to teach soft skills in many different venues. One was 
its ROTC program, which offered an elective open to the whole school in which students learned 
sound reasoning and problem-solving skills. The school rewarded students who demonstrated 
character and went “above and beyond the call of duty” with special t-shirts; students wore them 
with pride and, according to the principal, “really [pushed] hard to try to win” one. 

Team-building . We looked for evidence of teamwork during our classroom observations at the 
comparison sites. We saw no teamwork in the three academic classes we observed. Out of 19 
CTE class observations, less than one third captured examples of teamwork. The least successful 
was a highly unruly senior technical modeling class in which group work led to confusion, 
chatter, and bickering. The teacher appeared unable to channel students’ energy into productive 
pursuits, exclaiming, “You guys need to leam to work together.” Most examples of teamwork 
were positive, however; we saw students collaborating or leading their classmates to bleed 
brakes, prune shrubs, and prepare meals. A culinary class at Emerald generated particularly 
positive examples of students working together to solve unexpected problems. For instance, a 
team of boys making Oreo-based truffles for a buffet dinner ran into difficulties with the texture 
and consistency of their filling, and debated how best to scoop it into balls and then coat it in 
chocolate. They were able to use a melon bailer to make the truffles, then popped their tray into 
the freezer to render the balls firm enough to be coated in chocolate. Students in this class also 
sought out tasks if they were idle: While the balls of filling were in the freezer, a student from 
this team volunteered to take care of the sweetened and unsweetened tea for the dinner. 

Critical thinking and problem-solving skills. At Neon, the district’s arts school, an apparel 
teacher described how in learning how to conceptualize and depict a three-dimensional objects 
onto two dimensions (i.e., paper patterns for clothing), her students were learning the “same 
higher-level thinking skills that an engineer is getting.” This teacher also prompted students to 
think seriously and sensitively about topics that would affect them or their families in their future 
lives. In one class we observed, her students discussed ways in which the clothing they made 
could be adapted to disabled or elderly clients; they also researched the costs and benefits of 
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cloth versus disposable diapers, then discussed their findings and opinions with each other. In 
these discussions, the teacher urged her students to reflect critically on their personal beliefs. 

Teaching soft skills through CTSOs. During the worst years of the recession, the district cut 
back in many areas, but continued its support of CTSOs at all of its high schools. Scheduling 
constraints and transportation issues sometimes made it difficult for students to participate. 
Indigo administrators noted that their four-by-four block schedule, in which students took a full- 
year course in one semester, made recruitment tough. Students were not motivated to join or 
remain in a CTSO if they were not enrolled in a related class. Despite their scheduling 
challenges, Indigo’s CTSOs related to its marketing, business, construction, health sciences, 
family and consumer sciences, and horticulture programs were popular and well-supported. In 
particular, Indigo’s principal described the school’s FFA chapter as “vibrant” and “civic- 
minded.” Indigo’s horticulture teacher said that most of his students participated in FFA and 
gladly gave of their Saturdays and summers to plan for and attend competitions. When new 
students learned that their teacher came to school on the weekend to catch up on landscaping 
work or work with students on their FFA projects, they began to show up on weekends, too. 
Soon, these students were also helping beautify the school and engaging in FFA projects. 

Emerald offered CTSOs related to its health sciences, culinary, family and consumer sciences, 
and PLTW programs. Its culinary program was so popular and so successful at FCCLA 
competitions that many more students sought to participate than could be accommodated. The 
culinary program won several state competitions as well as national silver and gold medals. 
Articles and photos of past winners covered a “Wall of Fame” in the school’s cafe, and many 
students held state leadership positions. Upper-level culinary students completed FCCLA 
projects as part of the curriculum, including self-awareness exercises. 

The local community college. Interviews with ECC administrators, faculty, and counselors 
revealed mixed perceptions regarding the college and career readiness of East District graduates. 
One instructor believed that, overall, East students would benefit from more “real-world 
experiences” and “real-world type projects [like those] we would do in our applied science 
programs,” especially those activities that taught teamwork, research, and critical thinking skills. 
These skills were seen as helping East graduates avoid the “cultural shock” that some 
experienced in the much less prescribed world of higher education. 

National, State, and/or Local Assessments Provide Ongoing Information on the Extent to 
Which Students Are Attaining the Necessary Knowledge and Skills for Entry Into and 
Advancement in Postsecondary Education and Careers in Their Chosen POS 

Technical Skills Assessments 

As with core academic course assessments, CTE and POS end-of-tenn assessments were 
managed at the state level and administered via an online system. Most exams were developed 
by educators with input from industry, but some were industry-developed (e.g., A+, Cisco, 
Oracle). Other certification exams that served as end-of-course exams included the CNA for 
health sciences and the NCCER for construction. 
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Blue’s principal believed that, compared to his peers at the comprehensive high schools, his job 
was more difficult because his CTE assessment scores had to be as high as his core academic 
scores, in part to justify the additional resources needed to support his facility. This principal also 
used test score results to allocate resources. 

As previously noted, during our study, the state moved its CTE assessments to an online 
platform that quickly delivered student performance data to teachers. Broken out by student 
demographics and by classroom, these data allowed teachers to identify and address learning 
gaps at the question level. Teachers within programs could analyze why students missed certain 
questions and adjust their curriculum to target these gaps. Teachers worked in professional 
learning communities to compare outcomes; those identified as more effective at teaching certain 
skills mentored their peers in pedagogies and practices that better taught those skills. 

At some schools, the shift to an online testing platfonn was hampered by outdated technology 
and/or insufficient numbers of computers. Other issues hindered teachers’ ability to use the 
system. CTE teachers told us that there were limits to the numbers of test items that would 
populate on tests they wanted to give. They were also not able to score their own tests; instead, 
they were given to a school testing coordinator. Further, the test item bank from which end-of- 
tenn exam questions were drawn was only accessible on the day the exam was given. If a new 
course did not yet have an end-of-tenn exam, districts and schools were required to develop one 
with industry input. 

Teachers found ways to use the system to their students’ benefit. A career coordinator at one 
comprehensive high school reported that teachers discovered that if they created interim 
assessments from the item bank, their students performed better on end-of-tenn exams because 
the questions were similar. In most cases, teachers pulled their own interim test questions from 
the system’s test item banks; at Heliotrope, however, CTE teachers from the same program area 
were working to create common interim assessments. During weekly after-school meetings, 
Heliotrope teachers analyzed student performance outcomes by pooling the results from their 
shared interim assessments and identifying questions that caused problems across classes. 
Teachers then collaborated on strategies to improve instruction in those curricular areas. In some 
cases, teachers repeated instructional units until their students achieved mastery; in others, 
teachers who were more successful at teaching certain content would share their teaching 
strategies or offer tutoring to small groups of struggling students from other teachers’ 
classrooms. Heliotrope teachers also designed common wann-up activities. 

Not all interviewees reported such positive experiences with or perceptions of the online system. 
Some teachers told us they feared that their students’ scores were too accessible to department 
chairs, career coordinators, principals, and district and state CTE administrators. Administrators 
not only had access to reports on students’ interim and end-of-tenn assessments, they also saw 
the growth attributed to the individual teacher. 

Creating the Standards 

Industry representatives were highly involved in aligning curricula and assessments with 
industry standards. To secure state approval for new curricula or assessments, districts and 
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schools had to prove that their review board included postsecondary partners and at least 80% 
industry representatives. One teacher believed that this requirement placed onerous demands on 
program advisory boards, especially those with several new courses to review. 

As previously described, the state required CTE teachers to teach the academic skills embedded 
in their programs. Although state assessments did not explicitly test such academic skills, many 
test items could not be answered without mastery of these skills. The same was true for technical 
standards, which were laid out as task lists of competencies to master: In order to earn a CTE 
competency certificate, students had to master both academic and technical standards. 

Authentic Assessments 

Because the state’s technical skills assessment system was housed online, tests had to be 
delivered in an easy-to-administer, easy-to-score format. As such, all technical skills assessments 
were multiple choice and contained no applied assessments or demonstrations of proficiency. 
Many teachers and administrators expressed unhappiness with these assessments and what they 
perceived as an over-emphasis on rote memorization and written assignments. To counteract this, 
one CTE teacher told us that he made class labs count for a larger part of students’ grades. 

One principal believed that multiple-choice tests lacked authenticity and needed to be 
supplemented with CTE- or POS-specific capstone projects emphasizing real-world applications 
of academic and technical skills. As an example, he envisioned a horticulture student being asked 
to develop a complete prospectus for a landscaping job that would involve math, science, 
writing, and literacy skills. The student would design the project, select its raw materials, cost it 
out, and write a “nice proposal” for a prospective customer. 

Assessments were most authentic at Neon, where it was not surprising to see dance performances 
or student monologues supplementing the state’s multiple choice end-of-term exams. 

East’s CTE director acknowledged that although the current technical skills assessment system 
did not include a performance component, future iterations of these assessments might be able to 
include one. For the time being, CTE and POS teachers were expected to assess student 
perfonnance in an ongoing, formative way. 
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CHAPTER FIVE: South District 

POS as a means of answering the call of business and industry for better prepared students 


Abstract 

South District is located in a state that strongly encouraged high schools to develop career 
academies as a way to engage students and lead them to high-demand, high-wage careers. 
Through this and other policy measures, such as mandating articulation agreements and dual 
enrollment opportunities, and including industry-recognized credentials as part of college-and- 
career-ready accountability program, various parts of the state, including South District, were in 
essence implemented POS earlier than other parts of the country. 

Rather than create POS high schools, South District ensured that all students had the opportunity 
to participate by developing POS in every high school. Some of these POS were accessible 
through the district’s lottery system, and others were available for students attending their zoned 
high schools. There were no systematic differences between these programs. 

We planned to test an instrumental variable model of the effects of POS enrollment and CTE 
credit earning on GPA and graduation. However, similar to East District, POS enrollment in 
South did not generate substantial exogenous variation in number of CTE credits earned, which 
precluded us from interpreting the results of the instrumental variable analysis. We tested a 
conventional mediator model and found that students who enrolled in POS were 9% more likely 
to graduate and had slightly but significantly better GPAs than students who enrolled elsewhere. 
Both of these effects were attributable to increased earning of CTE credits. These results suggest 
that enrollment in POS may have benefited students in terms of retention and also achievement. 

We examined policy-relevant questions comparing the subsample of students who completed a 
POS to those who completed a CTE concentration, and the rest of the sample who did neither. At 
South District, 58.0% of the intervention group completed a POS. We found that more students 
in the rest of the sample earned AP credits and accrued college credits than their comparison 
groups, perhaps a function of the types of programs that constitute the other choice programs in 
South District (i.e., strong college preparatory programs such as International Baccalaureate). 

In order to assess the effect of completing a POS on high school achievement, we performed 
posthoc regression analyses on POS-relevant achievement measures for POS completers 
compared to the rest of the sample and compared to CTE concentrators. We found that compared 
to the rest of the sample, South POS completers were significantly less likely to earn more 1) 
STEM credits or 2) AP credits, but they were more likely to have a higher CTE GPA. The model 
for dual enrollment indicated a weak fit to the data and suggested that important factors in dual 
enrollment participation were not captured. 

The results comparing POS completers with CTE concentrators were similar: POS completers 
were significantly less likely to earn more 1) STEM credits or 2) AP credits, but they were more 
likely to have a higher CTE GPA. Again, the model for dual enrollment indicated a weak fit to 
the data and suggested that important factors in dual enrollment participation were not captured. 
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Unlike the other two districts participating in this study, no senior exit survey was conducted at 
South District. However, we were able to collect some industry-recognized credential data from 
South, which was not available at the other districts. The data were not complete, but among the 
students known to have sat for a credentialing exam, nearly 94% passed and earned the 
credential. Over three quarters of those credentials were earned by POS completers. 

The chapter concludes with a qualitative analysis of POS implementation in South District, based 
on the elements of the law mandating POS and the subsequent policy guidance framework 
provided by the funding agency (USDE/OVAE, 2010). 

Introduction to South District and the Intervention POS 

South District is a geographically large district with urban, suburban, and agricultural regions. 
South has been recognized for being in the vanguard of education reform, data-driven decision- 
making, and implementing career academies 6 . Although there were no wall-to-wall POS high 
schools in this district, there were instead career academies located in most comprehensive high 
schools. Many career academies were accessed through South’s school choice program, in which 
students applied to a lottery to attend POS, performing arts, International Baccalaureate (IB), and 
military ROTC programs. 

In the early 2000s, the district saw substantial growth in student enrollment, and the county 
passed a temporary sales tax increase to support school construction and renovation. Six high 
school campuses were augmented with state-of-the-art career academy facilities as a result of 
this revenue. 

South District’s state has an active legislature with respect to education. The state encouraged the 
development of career academies. The mandate fit South District’s goals of improving 
achievement, promoting diversity, and providing a specialized curriculum to engage students. At 
the same time, the business community was calling for better-prepared students, and district 
leadership believed that in addition to making education more rigorous and relevant, POS in the 
form of career academies promised to further their goal of having students graduate, as one 
teacher put it, “with a diploma and something else” (i.e., college credits, an industry-recognized 
credential). 


High Schools 

Unlike West and East District, POS were not limited to specific POS high schools; most of the 
high schools in South District contained career academies. In fact, in addition to the lotterized 
POS that were part of the district’s choice program, many high schools also offered career 
academies for their zoned students as well. Although the curriculum was not exactly the same 
between lotterized and zoned high school POS, those differences were not systematic, and in 
fact, lotterized POS across high schools also differed in scope and sequence. 


6 All career academies do not necessarily meet POS requirements but the career academies of South District did. We 
will use the terms interchangeably with South. 
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Due to the late entry of South District into the study, we were only able to conduct one site visit 
and thus did not visit many high schools. Although we tried to visit a wide range of schools, we 
were not allowed to visit schools that were in danger of not meeting their yearly academic 
progress goals. The schools we did visit had for the most part energetic POS teachers, often 
hailing from industry, who worked well with the community in terms of student internship 
opportunities. 


How POS Culture and Identity Engaged Students 

Rather than discuss school culture in a district with no wall-to-wall POS high schools in which a 
“POS school culture” might emerge, we comment upon how the individual POS foster their own 
culture and a sense of identity within the comprehensive high schools. The main means by which 
the POS engaged students was naturally through the curriculum content, because students had 
selected their POS. 

Although more prevalent in the past than during the time of our study, student cohorting was 
another way that the POS fostered a sense of identity among its students. Before new state 
mandates made cohorting difficult to impossible, most POS in South District kept their POS 
students together for academic courses. During the study period, some high schools cohorted 
POS students into science, English, and social studies classes, others limited cohorting to English 
and social studies only, and other schools had eliminated cohorting altogether. Math was not 
cohorted due to the wide range of levels of math into which students were placed. At one school 
we visited, the principal lamented the loss of cohorting, recalling the “family atmosphere” of 
students travelling together from class to class. In the large high schools of South District, having 
a “family” to travel with was one way of developing unique cultures and identities for each POS. 
Cohort teachers reported meeting together to better monitor student progress. 

Another means of developing student identity around a POS was through CTSOs, which were 
very popular among students across the high schools we visited. CTSOs provided students 
opportunities to develop leadership skills both through competitions and also through community 
service. FBLA students at one high school had adopted a kindergarten class at a local elementary 
school and they enjoyed it at least as much as the youngsters did. Health careers students in 
HOSA put on health fairs for the community, helping to check blood pressures and glucose 
levels. 

Finally, POS identity developed around program pride, whether in CTSO awards or general 
pride of purpose. In one classroom we visited, students wore T shirts declaring their membership 
in that “Triple A Magnet” POS. Every possible space on the walls of the class contained student- 
made posters, including along the top of the walls, just below the ceiling. Some contained the 
student’s name as the most prominent element, and then what they liked: basketball, FBLA, 
senior year, SAT, and colleges they wanted to attend. Bulletin boards were decorated with 
photographs of students at competitions. 
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Instrumental Variable Sample Description 

The South District portion of the study examined data for 3,506 secondary students, 1,985 of 
whom enrolled in a POS, while 1,521 were not. Table 5.1 displays descriptive statistics for the 
South District imputed sample. 

Table 5.1 


Descriptive statistics for imputed sample, South District 



(A =3,506) 

% 

Gender 

Male 

1,664 

47.5 

Female 

1,842 

52.5 

Race/Ethnicity 

Black 

668 

19.1 

White 

1751 

49.9 

Latino 

754 

21.5 

Asian 

185 

5.3 

Native American 

19 

.5 

Other 

129 

3.7 

F/RL eligible 

1,012 

28.9 

Limited English Proficient (LEP) 

176 

5.0 

Has Individual Education Plan (IEP) 

158 

4.5 

Number of Days Suspended 

No days 

3,264 

93.1 

1 or 2 days 

122 

3.5 

3 to 10 days 

113 

3.2 

More than 10 days 

7 

.2 

Graduated on time (2012) 

3,296 

94.0 


Mean 

SD 

Grade 8 pre-test math scale score 3 

354.77 

36.86 

Grade 8 pre-test reading scale score 3 

338.30 

42.22 

Grade 8 pre-test science scale score 3 

345.84 

57.88 

Number of GTE credits earned 

4.84 

1.57 


Note. F/RL = free/reduced price lunch program. Due to incomplete withdrawal data, we could not determine an 
attrition rate (cf. Appendix A). 
a Variable was grand mean centered. 


Missing Data 

We carried out a missing data analysis for South District and observed that the percentage of data 
missing for the variables ranged from 0.0 to 46.4%, with an average of 7.4%. We rejected the 
assumption of MC AR given mean differences in eighth-grade standardized test scores between 
those who were and were not missing GPA data for later years. We made the more relaxed 
assumption of MAR and handled missing data using the same procedure as reported for the two 
other districts. 
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Instrumental Variable Estimation 


We tested our hypothesized model across the imputed data sets and observed that it was over- 
identified with 9 degrees of freedom and fit the data marginally well (e.g., % = 65.38, CFI = .93, 
TLI = .74, and RMSEA of .04). We tested the model using a single imputed set to obtain a test 
and modification indices. We observed a statistically significant global test statistic (p < .001), 
suggesting that the model did not fit the data exactly (see Bollen, 1987, pp. 263 - 269). We 
observed modification indices and standardized residuals suggested that number of CTE credits 
earned should be regressed on free lunch status, reading score, and Black. We added these 
specifications and observed that the resulting model was over-identified with 9 degrees of 
freedom and fit to the data was generally good (e.g., 'i = 22.33, CFI = .98, TLI = .91, and 
RMSEA = .02). However, we tested the model using a single imputed set and observed a 
significant global test statistic (p = .007), suggesting that we should reject the hypothesis that the 
model fit the data exactly. 

With residual covariances in place, enrollment in a POS did not appear to have a significant 
effect on the number of CTE credits students took. As with East District, this result implied that 
enrollment in South District POS was not a good instrument for number of CTE credits earned. 
Thus, we did not interpret the results of this model. Instead, we tested a conventional mediator 
model with no residual covariances. This model was over-identified with 1 1 degrees of freedom 
and fit to the data was excellent (e.g., "i = 12.44, CFI = 1.00, TLI = .99, and RMSEA of .01). We 
tested the model using a single imputed set to obtain a % test and modification indices. We 
observed a statistically non-significant global test statistic (p = .33), suggesting that we should 
not reject the hypothesis that the model fit the data exactly (see Bollen, 1987, pp. 263 - 269). In 
addition, modification indices and standardized residuals did not indicate that any areas of 
substantial strain existed for the model. 

For this model, enrollment in POS predicted an average of only .70 more CTE credits earned and 
the model explained just 14% of the variance in credits earned (see Figure 5.1). Controlling for 
background variables, the indirect effect between enrollment in POS and GPA was b = .04 (p = 
.005, fl = .02), while the indirect effect between enrollment and graduation was b = .24 (p = 

.001). The latter effect is a probit coefficient and can be interpreted as indicating that holding the 
background variables constant, students in POS were 9% more likely to graduate on average. 

The direct effect between CTE credits and GPA was b = .05 (p < .001, P = .10), while the direct 
effect between credits earned and graduation was b = .34 (p < .001). The latter suggests that each 
additional CTE credit earned by students was associated with a 4% increase in the probability of 
graduation. In addition, each additional CTE credit earned was associated with a .05 unit 
increase in GPA. Taken together, these results indicate that (1) students who enrolled in POS 
were 9% more likely to graduate than students who enrolled in other programs, (2) students who 
enrolled in POS had slightly better GPAs than students who were not enrolled in POS, and (3) 
these effects may be attributable to increased earning of CTE credits. However, we note that this 
model did not correct for endogeneity so causal inference must remain tentative. 
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Figure 5.1 

Final model for South District 



Note: Significant unstandardized estimates are displayed. Covariates are omitted for conceptual clarity. 

In addition to the effects described above, several covariates were significant predictors of 
number of CTE credits earned, GPA, and graduation. Gender (i.e., being male), F/RL, and 
discipline occurrences had significant and small negative effects on GPA, while LEP, Reading 
Score, and Science Score had significant and small positive effects on GPA (ps < .001). In 
addition, Math Score had a significant moderate and positive effect on GPA (ps < .001). F/RL 
and discipline occurrences had significant and small negative effects on graduation, while Black 
had a significant and small positive effect on graduation (ps < .001). In addition, Reading Score 
and Math Score had significant and moderate positive effects on graduation (p < .001). Finally, 
F/RL and Black had small but significant positive effects on number of CTE credits earned, 
while Reading Score had a small but significant negative effect on number of CTE credits earned 
(p < .001). For a complete description of the estimates for South District, see Appendix A. 

Assessing the Effect of Completing a POS on High School Achievement 

As with the other participating districts, we employed two approaches to data analysis at South 
District. Here we present multiple regression analyses, using a different sample — POS 
completers, all other students, and CTE concentrators. All of the regressions excluded students 
who (1) withdrew or were missing course data and (2) were missing baseline achievement data. 
We were consistent with this even with the CTE GPA analysis (that does not control for baseline 
achievement) so that the regression sample would match the descriptive statistics to the extent 
possible. The sample for the CTE GPA analysis is further reduced to those who actually took a 
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Table 5.2 


Descriptive statistics, posthoc sample, South District 



Total 

(n = 2,488) 

POS 

Completers 
(n = 956) 

All Others 
(n = 1,532) 

CTE 

Concentrators 
(n = 396) 

POS 

Completers 
Compared 
to All 
Others 

POS 

Completers 
Compared to 
CTE 

Concentrators 

Characteristics 







Male 

46.3 

49.0 

44.6 

47.0 



Black 

20.1 

19.7 

20.3 

23.2 



White 

46.4 

46.5 

46.3 

48.0 



Latino 

22.8 

25.3 

21.3 

20.5 

* 


Asian 

6.2 

3.9 

7.7 

6.3 

*** 


Other/Multiracial 

4.5 

4.6 

4.4 

2.0 


* 

F/RL eligible 

30.7 

34.4 

28.4 

35.9 

** 


LEP 

5.7 

6.6 

5.2 

6.8 



IEP 

4.7 

5.4 

4.2 

5.8 



Grade 8 pre-test reading 

337.09 (43.45) 

327.44 (36.88) 

343.11 (46.08) 

333.32 (41.11) 

*** 

* 

Grade 8 pre-test math 

354.20 (37.60) 

348.47 (32.84) 

357.78 (39.88) 

350.40 (35.26) 



Discipline events 







None 

93.7 

93.7 

93.7 

94.9 



1-2 events 

3.3 

2.8 

3.5 

2.8 



3+ events 

3.0 

3.5 

2.7 

2.3 




Note. Excludes students who withdrew or were missing course data. Standard deviations of continuous variables are included in parentheses. 
$ * * * * 

p < .05, p < .01, < .001. 
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Table 5.3 


Descriptive outcome statistics, posthoc sample, South District 



Total 

(n = 2,488) 

POS 

Completers 
(n = 956) 

All Others 
(n = 1,532) 

CTE 

Concentrators 
(n = 396) 

POS 

Completers 
Compared 
to All 
Others 

POS 

Completers 
Compared to 
CTE 

Concentrators 

Percent Earning Outcome Credits 






STEM 

100.0 

100.0 

99.9 

100.0 



AP 

72.5 

63.9 

77.9 

70.7 

*** 

* 

College (accrued) 

4.2 

3.3 

4.7 

3.3 



CTE 

100.0 

100.0 

73.7 

100.0 



Outcomes 

M(SD ) 

M (SD) 

M (SD) 

M (SD) 



Overall GPA 

3.30 (0.70) 

3.21 (0.59) 

3.35 (0.76) 

3.22 (0.68) 

*** 


STEM credits earned 

5.32 (1.83) 

4.94(1.48) 

5.57(1.98) 

5.19(1.66) 


** 

AP credits earned 

3.20 (3.15) 

2.27 (2.66) 

3.78 (3.29) 

3.15 (3.11) 


*** 

College credits accrued 

0.06 (0.34) 

0.04 (0.23) 

0.08 (0.40) 

0.06 (0.34) 

** 


CTE GPA 

3.42 (0.65) 

3.49 (0.48) 

3.36 (0.76) 

3.41 (0.60) 


* 


n = 2,085 


n = 1,129 





Note. Excludes students who withdrew or were missing course data. STEM credits equals credits in science and higher math courses (i.e., above Algebra II). 
College credits accrued refers to the potential college credits associated with dual enrollment participation in high school. 

rfc * * * 

p < .05, p < .01, < .001. 
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CTE course among the student in the All Others group. Table 5.2 provides descriptive statistics 
for the posthoc regression sample. 

We found that of the 1,985 South intervention students, 1,152 (58.0%) completed a POS. We 
then identified the number of students who earned credits in the outcome measures defined in the 
Method chapter (i.e., measures of academic rigor, technical rigor, and transition as outlined 
there). Descriptive results of student course-taking are shown in Table 5.3. 

The following are South District’s results of posthoc analyses of high school achievement, 
controlling for background characteristics (full regression tables in Appendix A). Comparing 
POS completers to the rest of the sample, the results for STEM credits earned and AP credits 
earned are significant favoring the rest of the sample. The effect size for AP credits earned make 
it a substantive finding as well (-0.27, cf. Table 5.4). The results for CTE GPA are positive for 
POS completers and statistically significant; however, the poor fit of the model to both the CTE 
GPA and college credits data make the findings questionable. 

Table 5.4 

Regression results, POS completers versus all others, South District 

South District POS Completers Versus All Others 


Outcome 

tt 

Adjusted 

R 2 

b 

SEb 

fi 

P 

95% C7 of b 
lower upper 

ES 

Overall GPA 

2,488 

0.464 

.022 

.022 

.015 

.306 

-.020 

.064 

0.03 

STEM 

credits 

earned 

2,488 

0.372 

-.329 

.061 

-.088 

.000 

-.448 

-.210 

-0.19 

AP credits 
earned 

2,488 

0.457 

-.220 

.095 

-.035 

.000 

-.988 

-.605 

-0.27 

College 

credits 

2,488 

0.040 

-.015 

.014 

-.021 

.292 

-.043 

.013 

-0.06 

accrued 

CTE GPA 

2,085 

0.092 

.131 

.027 

.101 

.000 

.078 

.185 

0.20 


Note. All coefficients rounded to two digits. Full regression tables are found in Appendix A. CI= Confidence 
Interval. ES = effect size (Hedges’s g ) 


Table 5.5 shows the comparison of POS completers and CTE concentrators. The results for 
STEM credits earned and AP credits earned are negative and statistically significant (i.e., 
favoring CTE concentrators), and the results for CTE GPA are positive and statistically 
significant, but none of these results achieved substantively important effect sizes. The adjusted 
R 2 for college credits accrued and CTE GPA shows a poor fit of the model to the data, make the 
findings questionable. 

Unlike at the other participating districts, no senior exit survey was conducted at South. 
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Table 5.5 

Regression results, POS completers versus CTE concentrators, South District 

South District POS Completers Versus CTE Concentrators 

Adjusted 95% Cl of b 


Outcome 

n 

R 2 

b 

SEb 

fi 

P 

lower 

upper 

ES 

Overall GPA 

1352 

0.376 

.042 

.029 

.031 

.155 

-.016 

.099 

0.07 

STEM 

credits 

earned 

1352 

0.305 

OO 

OO 

.077 

-.055 

.015 

-.339 

-.036 

-0.14 

AP credits 
earned 

1352 

0.388 

-.608 

.133 

-.098 

.000 

-.869 

-.346 

-0.23 

College 

credits 

1352 

0.052 

-.008 

.016 

-.013 

.619 

-.039 

.023 

-0.04 

accrued 

CTE GPA 

1352 

0.053 

.073 

.030 

.064 

.016 

.013 

.132 

0.14 


Note. All coefficients rounded to two digits. Full regression tables are found in Appendix A. CI= Confidence 
Interval. ES = effect size (Hedges’s g ) 


Industry-Recognized Credentials 

Industry-recognized credentials and state licensure are two ways that potential employees have 
of signaling their work readiness (Bartlett, 2002). Some licensures, such as the CNA (Certified 
Nursing Assistant) or cosmetology licenses, have been offered in high schools for years. In 
addition, more certifications are being offered at the high school level, including at our three 
districts. However, reliable data on this measure has been as difficult to obtain at the high school 
level (Castellano, Stone, & Stringfield, 2006) as it has been nationally (Ewert & Kominski, 
2014). Our districts were no exception: We were unable to collect any data on student credentials 
at West or East. At South, we received some data on the students who took exams in order to 
earn an industry-recognized credential. These data were not collected in ways similar to other 
district data, and they appeared to be incomplete, although district personnel could not elaborate 
on the ways in which they might be. But we report on partial results in order to illustrate student 
participation. 

According to South District records, of the 2,488 students in our posthoc analytic sample, 390 or 
15.7% took an exam to earn an industry-recognized credential. Of these students who sat for a 
credentialing exam, 366 (93.8%) passed and earned the credential. Over 75% of those credentials 
were earned by POS completers. The most popular credentials earned were Certified EKG Aide, 
Certified Medical Administrative Assistant (CMAA), and Autodesk Certified Associate. 

Adherence to the Components of POS 

South District’s POS — structured as career academies in all high schools with little perceivable 
difference between lotterized (“choice”) and non-lotterized programs — placed a heavy emphasis 
on offering vertically aligned, progressively more intensive POS course sequences and rigorous 
college-preparatory academics combined with work-based learning, clinical experiences, dual 
enrollment and other postsecondary credit-earning options, online career exploration tools, 
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abundant college and career guidance, and industry-recognized credentials. Perhaps due to the 
mandated career academy opportunities in their high schools, most participants held college 
readiness and career readiness in largely equal esteem, defining these concepts in tenns of state 
and district policy guidelines. 

Although located in a state that exerted considerable control over educational policy, including 
CTE curricula and standards, South District administrators still enjoyed some flexibility in 
developing, implementing, customizing, and marketing their POS and career academies in 
conjunction with their postsecondary and business and industry partners. State and district 
policies both helped and hampered this work, and as in the other districts participating in the 
study, the economic recession frequently hindered the ability of schools and specific programs to 
offer the full range of experiences and resources required in robust POS. Despite their limited 
budgets, however, the schools we visited reported receiving unstinting support — material, 
financial, and personal — from their postsecondary, business, and community partners. Support 
also flowed in the other direction, as schools made significant investments in the communities 
they served, offering not just their facilities but also the professional and personal skills of their 
students and faculty. 

Participants almost unanimously agreed that South District supported student services and 
quality instruction in both academic and POS classrooms with frequent professional 
development. Participants were less sanguine regarding the amount of support — time, resources, 
and facilities — they received to implement both the state’s online high-stakes tests and the exams 
associated with industry-recognized credentials, which this state treated as its technical skills 
assessments. Finite resources also constrained but did not dampen schools’ efforts to engage in 
cross-curricular instruction and project-based learning; we saw evidence of both at all visited 
sites. Students in South’s career academies learned soft skills through classroom instruction, 
internship and job shadowing experiences, and voluntary participation in CTSOs. 

Incorporate and Align Secondary and Postsecondary Education Elements 

South District personnel led the process of collaborating with South Community College (SCC) 
administrators to develop opportunities for students to earn postsecondary credits while in high 
school. Faculty from both the district and the college gave final consent to the specific alignment 
of CTE curricula. In addition to credit transfer opportunities, the district and SCC partnered to 
offer an early admission option through which high school seniors could take their senior year 
courses at SCC instead of at their high school. Students had to enroll for a full-time course load, 
but courses were tuition-free. 

Professional development offered another avenue through which to blend postsecondary with 
secondary education. Large counselor turnover generated a constant need to provide professional 
development about available credit transfer opportunities to both high school and postsecondary 
counselors. Because the credits were not all automatically transcripted, counselors needed to 
identify potential credit opportunities and remind students to apply for them when necessary. 
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Include Academic and CTE Content in a Coordinated, 
N on-Duplicative Progression of Courses 


South’s aligned, non-duplicative course sequences consisted of at least three courses. The district 
described these sequences as roadmaps illustrating how academy programs included valuable 
industry-recognized credentials, work-based learning experiences, and credit transfer options. 
Nearly all academy sequences were enhanced by block scheduling, advisory board input and 
participation, and leadership and enrichment activities offered by related CTSOs. All sequences 
were vertically aligned with feeder middle schools, SCC, and other postsecondary institutions. 

Course sequences in both lotterized and non-lotterized POS were designed to lead to further 
education (often two- or four-year degrees) and well -paying jobs in high-demand areas. Some 
differences were evident in the entrance requirements for these POS, with some popular 
lotterized programs (e.g., medical) setting higher standards (e.g., achievement, attendance) for 
admission. At Scarlet, administrators dealt with space, class size, and hiring issues by treating 
their popular non-lotterized academies like their choice academies, requiring an application for 
admission. Outside the academy framework, “create-your-own” CTE course sequences were also 
possible. For those students who were unsuccessful in the lottery or unable to meet admissions 
standards for Plum’s automotive program, for example, administrators designed a “non- 
academy” pathway. For those students dissatisfied with their programs, change was possible only 
if open seats were available in another program. No changes could be made after the 10th grade. 
However, students in lotterized academies could choose to return to their zoned high schools. 

Plum’s academy coordinator reported that sequences were refined during year-end curriculum 
writing sessions in which teachers examined existing sequences and identified courses outside of 
CTE that would enrich and support the academy curriculum. In the criminal justice program, for 
example, “we... encourage [students] to take electives like psychology and sociology if their 
school offers it. If they’re looking for a science elective and their school offers forensics, we tell 
them to take those kinds of courses because although [they] are not CTE-related, they are subject 
areas that would help support someone looking to going into a criminal justice field.” 

Student Perspectives on Their Course Sequences 

Students interviewed during our senior- year site visit were well-informed regarding the structure 
and benefits of their POS course sequences, noting the increasing rigor and challenge of their 
programs, the industry-recognized credentials and dual credits they earned, and the work-based 
learning and CTSO activities available to them. Students also understood the lotterized versus 
non-lotterized nature of South’s academies. One senior engaged in the specialized phannacy 
technician program at Bronze described how she gained access to her non-lotterized medical 
academy after first gaining admission to the school — one of the district’s most competitive and 
well-regarded college -preparatory high schools — through its extremely popular lotterized STEM 
academy. Bronze’s medical academy was open to twelfth graders only after they completed a 
sequence of health science prerequisites. This type of “academy within an academy” structure 
was also present at Lavender, where two seniors in the drafting and design academy said that 
participation in a specialized house restoration project (a partnership between the school, a non- 
profit, and a local architectural design Finn) was predicated on completion of two years of 
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drafting or engineering courses. One of these students called the project “a pseudo academy” that 
entailed enrollment in a “very specialized class [in which] we’re learning specific industry 
construction techniques.” The other affirmed that it was only for experienced juniors and seniors, 
because it required a more “mature state of mind.” 

Students appeared to understand and appreciate the role of the academies as a place to explore 
and imagine different careers for themselves. A medical careers student said that his experience 
in the career academy had been “really good because I’ve learned that it’s really hard, especially 
when it comes to choosing what you actually want to do, because there’s so much to choose from 
[in the field of medicine].” He went on to say that being in a POS helped him and his peers 
decide if they really wanted to pursue medicine or not. 

A number of business academy students we met at Periwinkle said that they had college and 
career plans that lay outside their program areas but saw the application of their business studies 
to their future goals. One student indicated that the business academy had been her first choice of 
program; although she was interested in majoring in psychology and becoming a practicing 
psychologist, she believed that business skills could enhance her success in this field. 

Offer the Opportunity, Where Appropriate, for 
Secondary Students to Acquire Postsecondary Credits 


Dual Enrollment 

Free tuition and books were provided to high school students who dually enrolled in a specific 
list of postsecondary classes, whether they went to a community college or university. Nominal 
fees were associated with attending the state university. Although most students took dual 
enrollment courses on the SCC campus, South District teachers could be certified to teach these 
courses on the high school campus as part of their regular load. Some students preferred this to 
going to the college for the class. 

To be eligible, high school students had to pass all parts of a placement exam and otherwise 
show college readiness by having a 3.0 GPA (2.0 for CTE certificate dual enrollment courses). 
Students had to maintain a 3.0 GPA at the high school and at least a 2.0 on the college transcript. 
High schools had to give the same weight to college-level dual enrollment courses as AP courses 
when GPAs were calculated. This policy began to attract high-achieving students pursuing high 
GPAs who might not previously have been interested in SCC courses. 

The state created faculty committees to establish the equivalency of courses based on content, 
and the subsequent list of courses that high schools accepted as meeting their requirements 
applied to all high schools statewide. However, the schedule for CTE courses at SCC made it 
difficult for high school students to take them: Most were offered in 7-hour blocks, either in the 
daytime or at night. South District personnel decided that general education courses would be the 
most helpful to their students because they would allow them to graduate high school having 
completed one year of college general education requirements. Students were free to take courses 
not on the dual enrollment course list, but they did not enjoy the same free tuition, fees, or books. 
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Student participation in dual enrollment was one of the measures by which high schools were 
rated in the state’s school rating system. This created incentives to increase participation. One 
high school transported all eligible dual enrollment applicants to SCO to take the placement 
exam. At another school, students had a difficult time passing all parts of the placement exam. 
Staff there considered doubling up ninth-grade mathematics courses so that students would be 
ready sooner for the placement exam. 

In general, we found South District students were quite sophisticated in their knowledge of how 
to acquire college credits and which institutions would accept them. Many students appeared to 
have participated in dual enrollment starting in 10th grade and were graduating with multiple 
college credits, some even having earned an associate degree as well. Students told us that dual 
enrollment was heavily promoted. One high school estimated that 20% of its seniors took dual 
enrollment courses. This was on the high end; participation was not so high at other schools. 
Students learned about dual enrollment through the guidance department, which one counselor 
called “dual enrollment central.” Counselors disseminated infonnation about the program by 
visiting classes with PowerPoint presentations and holding meetings with eligible students. 

Articulation Agreements 

Articulation agreements were created at the district level, with input from the schools’ academy 
coordinators. Curriculum review committees made up of high school and postsecondary faculty 
reviewed assessments, textbooks, and projects and decided what could take the place of a college 
course. Every CTE program had at least one articulation agreement. 

Agreements were designed so that high school students who completed designated coursework 
and passed either an industry certification exam or a proficiency test would not have to take the 
related introductory course in college. However, the articulated credits were not immediately 
transcripted. Students had to enroll at SCC in a related program within two years in order to start 
the process of obtaining their credits. As the agreements developed, some postsecondary 
programs opted to administer challenge exams to students from lower-rated high schools as a 
means of ensuring that students knew the content. This led to a new policy in which all students 
applying for the college credit had to take challenge exams if a given program required them of 
anyone. Other requirements included presenting a portfolio or proof of having earned an industry 
certification. 

Given the way these two credit transfer programs were structured, South District students were 
encouraged to take advantage of both immediate and escrowed credits. One high school 
counselor described how she sold these opportunities: “Especially for our kids who are in the 
academies, it would make sense for them to get a concurrent [dual enrollment] credit now for 
something they’re learning in the academy. So we encourage those kids to go ahead and get your 
‘gen eds’ [general education requirements] out of the way through the dual enrollment, and then 
at the end you can also be able to pick up additional credits through the articulated process.” 
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Lead to an Industry-Recognized Credential or Certificate at the 
Postsecondary Level, or an Associate or Baccalaureate Degree 


Of the three districts participating in the study, none put greater emphasis on industry-recognized 
credentials than South, which bcnefittcd from the force of a fairly recent state law requiring 
districts with workforce development agencies in their area to develop career academy programs 
that allowed students to earn college credits and industry-recognized credentials and prepared 
them for well-paying, high-demand jobs. Although industry credentials were primarily targeted 
at POS participants, students taking a general course of study at a comprehensive high school 
could take a sequence of related courses and qualify to sit for a credential. The state made this 
initiative stick by supporting districts with resources, professional development and training, and 
an accountability component. During our study, student participation rates in courses offering 
industry-recognized credentials were scored in the school grading system, but participants told us 
that student success rather than mere participation would eventually be measured. As such, 
students’ failure to earn the certifications could negatively impact the school. 

South furthered state policy by redoubling its efforts with industry and postsecondary partners 
and by fully funding the cost of students’ certification exams — the only district to do so in the 
state. The district CTE administrator responsible for securing the state’s industry certification 
agreements for its medical and health science programs noted that both the state and the district 
recognized the value of industry-recognized credentials in attracting and retaining high-quality 
students. Industry credentials also offered value as sources of longitudinal achievement data: 
Some licenses and certificates, particularly in medicine, could be tracked into students’ 
postsecondary and career trajectories. 

In addition to signaling work readiness, industry-recognized credentials and related CTE 
program completer certificates were also associated with financial benefits. Plum’s academy 
coordinator stressed how these credentials were a powerful marketing tool for recruiting new 
students; in her outreach efforts to area middle schools, she talked to students about how “kids 
are graduating and walking right into jobs. Whether it’s something that they want to do full-time 
or not. . .we talk to them about how you’re [going to] have certifications and experiences, and 
while you’re in college, you can be using those to earn money in the field that you want to be 
in.” Given their vertical alignment with articulated college programs, many industry-recognized 
credentials were recognized for credit at colleges and universities. 

Accountability 

Interviewed principals frequently emphasized the importance of industry-recognized credentials 
to the district’s school grading system. Periwinkle’s principal outlined its components, which 
included graduation rates for academy versus non-academy students, state high-stakes test 
scores, college readiness indicators (SAT, ACT, state postsecondary readiness test scores), career 
readiness indicators (industry-recognized credentials, CTE completer certificates), and student 
participation in “AP, IB, and industry certification courses.” As Scarlet’s principal claimed, “the 
more certifications [students] get, the better it is for us.” Whereas a school’s grade used to be 
detennined largely by academic test scores (e.g., reading, writing, math, science), industry 
credentials were playing a larger role. Scarlet’s principal foresaw a shift in how these credentials 
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would be weighted in school grades, with more points awarded for passing tests than 
participating in courses. Although generally positive about this “critical piece of success of a 
school,” he saw the emphasis on industry-recognized credentials as creating certain dilemmas, 
particularly for CTE teachers. Once urged to assist academic teachers in reinforcing math, 
science, and literacy, CTE teachers were now being pulled in a different direction without a 
reciprocal expectation of support from their academic colleagues. 

Perceived Problems and Benefits 

Other problems associated with implementing industry-recognized credentials included the need 
to vertically align with both middle school and postsecondary curricula, program capacity issues, 
funding and scheduling constraints, problems securing computer labs or the latest versions of 
software needed to pass exams, and network or hardware glitches. Although the district used its 
bargaining power to secure the lowest costs for exams and sought site licenses whenever 
possible, costs for students to take tests remained quite high — more than $ 1 million district-wide. 
As a result, for some especially expensive exams, testing costs were only covered once per 
student. Further, a few participants believed that some credentialing exams were too difficult, 
most often citing as examples the ASE certification exam and the test for Final Cut Pro, the 
video-editing software. Others questioned whether certain credentials — all selected by the 
state — were relevant or appropriate to their academies. Plum’s academy coordinator relayed how 
her school’s criminal justice teachers initially “cringed” over the largely secretarial credential 
available in their program. Jade’s principal feared that some credentials represented “a 
disconnect” with students’ real needs. In the case of his school’s medical academy, he was 
compelled to add a class to support the CMAA credential, which he felt was too low-level: “My 
kids are more interested in pursuing careers being doctors, nurses, [and] pharmacists.” 

Overall, most participants believed the opportunities offered by industry-recognized credentials 
outweighed their drawbacks. Lavender’s academy coordinator cited the example of a medical 
academy student who graduated with 40 college credits, his CNA, CMAA, and EKG Technician 
certificates, and a state financial aid award. As she said, “That’s what our focus is, not one or the 
other, not college or career, but both. . . that’s what I always tell our students: ‘I’m here to open 
your options up, not to limit them. I want you to have [whichever] college you want to go to, not 
the ones that would accept you.’” A guidance counselor at Lavender saw POS as a means of 
breaking the cycle of poverty in this at-risk neighborhood. Many of her students’ parents had 
little fonnal education, and some undocumented families actively feared the occasional presence 
of police outside the school. Although she was pleased that both students and parents recognized 
the value of students’ experiences at Lavender, she noted that for those without documentation, 
participation in many credentialing exams and access to state financial aid and scholarship funds 
was impossible. A district CTE administrator confirmed that certain exams, particularly the 
CNA, required a Social Security number. 

Student Experiences 

The impact of South’s investment in industry-recognized credentials was nowhere more evident 
than in students’ comments about their POS experiences. At Plum, one automotive student had 
not only passed the difficult ASE exam, but was actively planning for the additional 
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postsecondary coursework and field experience he needed to become a master technician and be 
able to confidently say, “I have a patch that says, ‘I know what I’m doing.’” Another Plum 
student had already acquired his Allied Health, CPR, EKG Technician, HIPAA (Health 
Insurance Portability and Accountability Act), and lifeguard training certificates; he was also 
planning to sit for his phlebotomy certificate as soon as he turned 18. At Periwinkle, a student in 
the First Responder program successfully passed eight AP classes, obtained her CMAA and CPR 
certificates, and was waiting to hear about her admission to a pre-med program at a major state 
university. One Scarlet construction academy student showed us the OSHA card he carried in his 
wallet; another Scarlet student made a clear connection between his certification in Final Cut Pro 
and the baccalaureate degree he wanted to pursue in film and television. 

A student in the Bronze medical academy’s pharmacy technician program asserted that she was 
prouder of her internships and earned certificates — which included Allied Health, Basic Life 
Support (BLS), EKG Technician, HIPAA, and Phannacy Technician — than her AP credits. 
Inspired by a relative who had passed away after a battle with cancer, this HOSA member 
believed her certifications would help her pay for medical school and later become a cancer 
researcher. Her HIPAA certification had already helped her secure summer work in a law office. 

One senior sports medicine student at Lavender spoke of the hands-on, supportive quality of her 
program and how she had been able to earn 13 college credits, work toward her CMAA, EKG, 
and personal trainer certificates, and help real patients at the same time: 

The medical program has definitely put me a thousand steps ahead of everybody. I mean, 
it’s definitely a great opportunity. . . I wouldn’t trade anything that I’ve done in medical 
for anything because. . . it placed me on a pedestal that a lot of students going into the 
medical field haven’t had, you know? They may have graduated high school, but I can 
say I graduated high school with my CNA and my EKG and I worked towards my 
personal trainer. Even if I haven’t gotten it, I still have the background behind it. I think 
it’s a really strong support system, especially applying for college. 

State and Local Legislation or Administrative Policies 
Promote POS Development and Implementation 


State Policies 

State funding and other resources. This state provided ongoing funding specifically for CTE. 
Whereas the state funded six periods a day for high schools, South District chose to fund a 
seventh period in order to make it easier for students to participate in POS. Thus students who 
needed to double up on core classes still had room in their schedules for their academy courses. 

Establish formal procedures for the design, implementation, and continuous improvement of 
POS. This state exerted more control over education policy than did any of its districts or other 
local education agencies. The state approved all courses and assigned them a common course 
number. This helped students who moved within the state to pick up their studies where they left 
off. Although curricula were state -provided, they could be personalized according to area 
workforce needs, said one counselor. For example, some criminal justice programs focused on 
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corrections because there were prisons located nearby, whereas other programs addressed law 
enforcement. 

Ensure opportunities for any secondary student to participate in a POS. State legislation 
encouraged the development of POS as career academies, and every district had to offer at least 
one career academy. Most districts had more, and South had several POS in each high school. 

Require secondary students to develop an Individual Graduation or Career Plan. State law 
required students to make yearly updates to their individual graduation plans (IGPs), which 
began in eighth grade. The state’s online system functioned primarily as an electronic high 
school planner, with modules offering college and career infonnation and optional career interest 
inventories. One counselor said that although they lacked the time to meet individually with 
students to do those activities, they led some in classroom settings and urged students to explore 
the website on their own. The students we spoke with were largely unfamiliar with IGPs. 

Provide resources for the long-term sustainability of POS. The state funded K-12 education, 
which included support for POS. However, more than one person complained to us about 
“unfunded mandates” — new initiatives launched without the resources to support them. 

Other relevant state policies. This state passed a class size reduction law mandating that all core 
academic courses at the high school level contain no more than 25 students. This change 
reverberated through every high school’s offerings, and some electives, such as classes attached 
to student internships, had to be dropped. For example, whereas a biology class could have no 
more than 25 students, no such limits existed for a marine biology class — an elective. At one 
school we visited, a marine biology class had 45 students. The pressure to reduce class sizes 
affected the availability of other resources. Some IT students we interviewed complained that 
there were not enough computers in their academy classes because non-academy students were 
being placed in their classes because they had no size limit. One academy coordinator said that 
the law had also ended common teacher planning time and student cohorting. One culinary 
teacher said that she had so many students in her class that she could no longer participate in 
local and regional CTSO competitions. Schools were compelled to delay implementing new 
academies because moving academic teachers around complicated the entire schedule. Districts 
were fined if they did not meet the class size reduction law’s requirements. 

As noted, the state mandated that career academy programs lead to industry-recognized 
credentials; it also maintained a database of approved certification exams. The state purchased 
site licenses for some of these exams, which included practice tests. Some participants noted that 
serious consequences were attached to failing the exams. One teacher said that if his students did 
not pass their credentialing exams, his program could be dropped. Although there were no 
negative impacts to students’ grades if they did not pass their credentialing exams, the number of 
students taking industry-recognized credentials did fonn part of a school’s grade. 

Another noteworthy state policy involved online course-taking and exams. The state moved its 
high-stakes testing system to an electronic platform. As a result, computer or IT students were 
displaced from their classes so that other students could use school computers to take their state 
tests. Participants noted that newer high schools tended to have the newest computers, leading to 
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potential inequities if older facilities had technical troubles during testing. To address this, the 
district established some schools as district-wide testing centers. A counselor we spoke with said 
that she and other counselors were “tremendously involved” in online course-taking because they 
registered students for courses, communicated with online instructors if students were in danger 
of failing, and monitored the completion of courses affecting graduation. She laughed at the 
memory of being told that online courses “won’t be any work for you.” She noted that 
counselors serving the state’s online high school earned a higher salary than their school-based 
peers, although she acknowledged that the online counselors carried caseloads in the thousands. 

District Policies 

Local funding and other resources. South District’s POS received a high level of support from 
the schools that housed them, including the use of facilities, professional development for 
teachers, equipment, and funding — all of which came from state K-12 education funding. 
However, POS enjoyed other sources of support. Several years ago, district officials asked the 
county to levy a half-cent sales tax eannarked for school construction and improvement. The 
measure passed, and resulting funds led to the construction of many new high schools with state- 
of-the-art equipment for POS. Some of that equipment was provided by local businesses; for 
example, at Periwinkle, an area television station donated an on-air news desk and editing studio. 

In all schools, POS faced cutbacks during the recession’s worst years, and some people we spoke 
with feared the demise of the entire academy concept at the state level. As a cost-saving measure, 
the district considered eliminating academy coordinator positions. A drafting and design teacher 
said that his program might have to forego the use of the most current CADD software because 
they lacked the funds to upgrade the hardware needed to support it. More than one participant 
told us that their aging operating systems were incompatible not only with instructional software, 
but also with administrative software. Problems beyond those of funding affected schools’ use of 
technology: Students complained that they could not take their credentialing exams online 
because the district’s firewall prevented them from doing so. This same firewall forced teachers 
to download materials at home because they were unable to access them in class. 

Establish formal procedures for the design, implementation, and continuous improvement of 
POS. Although the state provided a common curriculum framework, POS were created at the 
local level by local secondary and postsecondary faculty who worked to create articulated, 
vertically aligned sequences of academic and CTE courses. Many schools built common teacher 
planning time into their schedules; many also cohorted students through both their POS and core 
academic courses. However, the state’s class size reduction law affected schools’ ability to 
maintain these structures. Some schools used district-sponsored professional development days 
to preserve time for teacher teams to work together. The law also led to larger CTE classes than 
were considered safe by the state’s own regulations, said some CTE teachers. 

Overall, despite policy and resource limitations, detennined school administrators found ways to 
support the career academy concept and POS in general. Periwinkle’s principal, who opened 
several high schools and created many career academies during his tenure, believed that well- 
designed POS required a combination of excellent teaching, continuous marketing and 
community outreach, and the resources to make it all work: 
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Well the first thing is — and again it goes back to my big three — you have to get teachers 
on board that are passionate about what they do and have knowledge of the industry or 
the curriculum that they’re teaching. Not just the book aspect of it but actually being out 
there in the real world. So staff is crucial. Number two is, you have to go out and recruit. 
You have to get your name out there, you have to sell your program, you have to get 
students who want to be involved in the program. The big thing is you don’t want to put 
kids in there just to fill up the class — that’s like the killer. And then number three is, you 
have to have the facility. And our district has been instrumental in putting programs in 
schools and giving them the facilities and equipment to be second to none. You’ve been 
through our school, you see what we have. It’s top-notch stuff. 

Ensure opportunities for any secondary student to participate in a POS. Whether a POS was 
lotterized or non-lotterized was detennined by the district, with some school input. Many POS 
changed their status during the four years of our study. As a number of participants explained, 
choice versus non-lotterized status and the purity of the lottery depended as much on politics and 
parental pressure as popularity. An academy coordinator at Periwinkle said that although all 
lottery and choice decisions were ultimately made at the district level, schools had some latitude 
in seeking “weighted” admissions for particular programs. For example, in the case of a program 
with low or no enrollments of minority students, the school could make a case that such students 
be given admissions priority. Schools could also seek to change the status of a POS. Jade’s 
principal said that they had successfully shifted two POS to non-lotterized status because they 
were already operating at “1 10% capacity” and preferred to fill seats with students zoned for that 
high school. Changes like these to any one high school had ripple effects across the district: As 
students moved from one school to another, so did teachers and funding. 

As Periwi nk le’s principal noted, recruitment was key to creating vital POS. Schools used a 
number of strategies to spark student interest in their programs, including district-sponsored high 
school showcases. Plum’s academy coordinator noted that industry-recognized credentials were 
a big selling point at their middle school recruiting events, to which the school sent high- 
achieving, personable students who could talk knowledgably about their college and career 
options. The district also conducted home visits to students from non-English speaking 
households to ensure proportional representation. 

Eligibility requirements for entry into the POS lottery were not onerous because the district’s 
goal was to encourage student participation. Some programs required a 2.5 GPA, but most only 
requested three favorable teacher recommendations. Some programs — like medical, veterinary, 
and computer science — required a 3.0 GPA, but had recently dropped Algebra I as an eligibility 
requirement. This move upset some teachers who felt it lowered standards. District personnel, 
however, noted that not enough middle schools offered Algebra in eighth grade. The district’s 
lottery included a preference for students who participated in a middle school magnet; some 
teachers believed that such feeder preferences limited opportunities for students with genuine 
motivation and interest. 

Provide resources for the long-term sustainability of POS. Although the superintendent and 
school board strongly supported POS, district GTE personnel recognized that new principals 
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needed to be educated and brought on board with POS and the career academy model. A district 
CTE coordinator said that she spent time visiting schools and principals and sharing the benefits 
of POS, especially their power to retain and recruit quality students: “We do a lot of educating.” 

One principal with a long history in the district spoke to us about her experiences as the principal 
of a high school without academies. At the time, she not only felt limited in the resources and 
opportunities she could offer students, but also in the quality of the student population she could 
attract. She started a POS at that school before leaving to accept a different principalship. 

Other relevant district policies. Students enrolled in lotterized POS who failed a POS course 
were given a chance to improve their grades during a probationary period in which they received 
extra resources and support. If they failed to improve, they could be sent back to their zoned high 
schools. 

Ongoing Relationships Among Education, Business, and Other Community Stakeholders Are 
Central to POS Design, Implementation, and Maintenance 

Business and Industry Partnerships 

South District’s commitment to offering industry-recognized credentials and work-based 
learning opportunities to students in its POS was greatly aided through the support — material, 
financial, and personal — of area business and industry. At every school we visited, participants 
described the many businesses that contributed equipment, money, time, and expertise to 
enriching their career-focused programs. Some businesses, both large and small, were able to 
offer both current and future employment and scholarship support. For example, the district’s 
phannacy technician program, available in some of its medical academies, was sponsored by a 
major national pharmacy chain; the chain provided materials and funding for the high school 
curriculum, sites for clinical placements, college scholarships, and guaranteed pharmacist jobs 
for successful graduates. 

Scarlet was one of two South high schools offering architectural design and construction 
programs that participated in the ACE Mentor Program, which brought schools, students, and 
industry professionals together for a year’s worth of concentrated lessons, teamwork, career 
exploration, and a hands-on construction project. One of the instructors in charge of the ACE 
program noted that at least 90 to 100 business partners actively supported the program through 
mentoring, advising, and donations. Both he and Scarlet’s principal believed that this support 
was a valuable way for businesses to invest in their future. During our site visit, at least a half- 
dozen of these partners — all enthusiastic — turned up to view the progress being made on the 
Habitat for Humanity house being built by Scarlet students in the school’s large and well- 
appointed construction lab. Beyond hands-on experience, the principal reported that students 
were graduating with OSHA cards, industry certifications, and job offers from area businesses 
that knew and valued the quality of their training and work. 

Schools also benefitted from the support of advisory boards. At Periwinkle, the principal noted 
that there were rarely fewer than 40 or 45 business and community partners present at their 
monthly meetings, including representatives from the hotel and tourism industry, television and 
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film, sports, and local restaurateurs. These partners, he said, had not only contributed materially 
to the school’s POS, but also offered job shadowing and internship sites to Periwi nk le students. 

Depending on the curricular or financial support being provided, the responsibility for managing 
business and industry relationships was shared among district personnel, administrators, 
academy coordinators, and teachers. In the case of the district’s medical academies, for example, 
the district coordinator described the process — repeated at two-year intervals — through which he 
worked with district lawyers to negotiate clinical experiences and internship placements for 
qualified students at over 100 sites. Placements were then facilitated by school-level academy 
coordinators and teachers for all students who sought them. All of the medical academy students 
we interviewed commented on the many and varied opportunities they had to work or job 
shadow in hospitals, walk-in clinics, nursing homes, sports medicine facilities, and other sites. At 
Plum, one local hospital not only served as a popular site for clinicals, it also underwrote a 
portion of the cost of equipping the school’s medical lab. Further, instructors in the medical 
academies were often experienced practitioners — we met doctors, nurses, and a fonner athletic 
trainer for an NFL team — who were able to draw on their extensive networks to bring students in 
regular contact with medical and public safety professionals, including doctors, nurses, EMTs, 
lifeguards, firefighters, HIV/AIDS experts, sports medicine providers, and recent graduates who 
had pursued medical studies. Although we focus here on medicine, we observed similar 
partnerships and opportunities across the broad range of career academies we visited in South. 

Community Partnerships 

Many interviewees at the schools we visited attested to the strong relationships between their 
schools and the local community. Geographically quite large, South served a highly diverse 
student population; many students were low income and came from a broad variety of cultural 
and language backgrounds. A number of participants commented on the district’s high numbers 
of undocumented students and parents and the great need to build trust between the school and 
the community by serving that community. Principals, counselors, teachers, and students all 
described participating in special events and extracurricular and co-curricular activities 
benefitting various local organizations, schools, groups, and families. At Plum, the school itself 
was a community hub. The school rented out its auditorium for events and hosted weekend 
church services; it also served as the site of an annual car show, health fairs, and regular sporting 
events: “If you come here at night, there’s tons of cars here. There’s always something going 
on. . . The community uses the school a lot,” commented one guidance counselor. At Lavender, 
one of the school’s language facilitators hosted a local radio program targeting a large migrant 
community; the host used the show to describe school opportunities. 

As noted, supported by mentors from the local business community, architecture and 
construction students at Scarlet were engaged in building a Habitat for Humanity house for an 
area war veteran and his family. Beyond their expertise, area businesses also donated materials 
and equipment for the house. At Lavender, students were working to redesign, restore, and 
convert a local historic property. Interviewees at several schools mentioned an initiative through 
which IT students refurbished and tested older computers to be donated to disadvantaged 
students and other community members; this initiative was supported by a local low-cost internet 
service that helped make online access affordable for low-income families. During our site visit 
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at Periwinkle, business students, many of them DECA members, had just returned from a 
Christmas party at a local elementary school; the students had adopted this school, purchased 
presents for all of its kindergarten students, and hosted a party complete with a student Santa and 
elves. One student ensured television coverage of the event by writing a press release and 
securing a film crew from a local news station. 

Postsecondary Partnerships 

In South District, articulation agreements with community colleges and state universities were 
coordinated largely at the state and district levels. Unlike West, where many of the CTE teachers 
we interviewed participated in advisory boards, fewer South educators reported working directly 
with postsecondary partners to align or develop curricula. However, we noted evidence of active 
involvement with local postsecondary partners at every school we visited. Teachers, guidance 
counselors, and students all spoke of abundant opportunities to participate in dual enrollment, 
including spending full days at SCC or a regional state university. Some teachers were 
university-credentialed to offer dual enrollment courses at the high school. Every site noted the 
active presence and participation of local and regional postsecondary institutions in college fairs, 
scholarship and financial aid workshops, admissions counseling, and other college awareness 
activities. Postsecondary partners participated on advisory boards. 

Sustained, Intensive, and Focused Professional Development Opportunities for 
Administrators, Teachers, and Faculty Foster 
POS Design, Implementation, and Maintenance 

Professional development was provided to all South school personnel (i.e., teachers, academy 
coordinators, guidance counselors, administrators) in a variety of fonnats and in a number of 
locations, including in school, online, and offsite at local postsecondary institutions. Some was 
delivered during a half-dozen dedicated full-day in-services distributed throughout the school 
year. At the high school level, additional professional development took place during once- 
monthly morning in-services; on these days, students arrived two hours late. District staff, 
invited experts, administrators, and occasionally other teachers were most often mentioned as 
delivering professional development. Much of the content of these trainings was district-driven 
and focused on state or district policy; college and career readiness initiatives; best practices in 
instruction, assessment, and classroom management; and hands-on analyses of assessment data 
from the state’s high-stakes tests. Principals were also given some latitude to deliver school- 
specific professional development on a range of subjects, including in support of POS. 

Career Academy-Focused Professional Development 

Training for career academy coordinators and teachers in South District was largely arranged by 
the district’s career academy specialists and targeted to the specific needs of each academy’s 
curriculum, assessments, technology, and certifications. Some of these trainings featured 
National Career Academy Coalition (NCAC) guidance on standards of practice. Schools with 
NAF academies participated in NAF trainings, including intensive summer institutes. 
Coordinator meetings occurred monthly and included some professional development, covering 
such topics as legislative updates, dual enrollment opportunities, industry certifications, CTE 
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completer certificates, and new graduation requirements. One new career academy coordinator 
reported that although she went through no formal preparation for her role, she had been 
mentored by a fonner coordinator and was learning a great deal from the district’s monthly 
meetings. One coordinator shared that she was participating in a six -week series of seminars on 
the effects of poverty; she learned about it through the district’s online professional development 
portal. The same coordinator mentioned that her POS teachers received professional 
development on fostering literacy and writing skills in the CTE classroom. 

In addition to regular in-services, POS teachers noted that they were able to participate in 
academy-related teacher externships and technology- or certification-specific trainings (e.g., 
Adobe, CISCO). Teachers themselves offered formal and infonnal professional development. 
One school principal noted that the success of his biotechnology program was due to one 
teacher’s gifts as a trainer: “When we were able to hire her, she was probably the best science 
teacher in our district and the one most ready to tackle a biotech program, and then she’s been 
able to train two other teachers to come along with it.” 

Professional Development for Guidance Counselors 

Guidance counselors reported that because of constrained staffing conditions at their schools, 
they spent much of their time scheduling courses and coordinating testing activities. As such, 
they reported that much of their professional development was focused on the state’s high-stakes 
testing system, electronic IGPs, and preparing for college entrance exams. As one guidance 
counselor noted: 

Most of our professional development these days seems to be related to [the state’s high- 
stakes test.] So all of us [participate]. All of us in every school and I don’t care what your 
job is. If you are a cafeteria manager. If you are a secretary. If you are. . . a high school 
guidance counselor, a career choice coordinator: We are all involved in professional 
development related to [the high-stakes test]. 

Another interviewee affirmed this focus on testing, acknowledging that, “a lot of the professional 
development, especially at the school site, is geared towards the teachers more so than 
counselors.” However, guidance counselors also mentioned participating in development focused 
on getting students college-ready and, to a lesser extent, career-ready. Interviewees spoke of 
trainings related to the state’s scholarship program, the state’s college-and-career planning 
website (underutilized by students, according to one interviewee), and college exploration 
activities (e.g., campus visits, college fairs). One guidance counselor reported that she was 
currently participating in college prep training offered by admissions counselors at a local 
university; this training was being paid for by the university and accepted for in-service credit. 

Counselors reported that professional development less often addressed counseling services for 
students with personal or family issues. One guidance counselor said she wished the district 
offered more support related to student mental health and substance abuse problems. Overall, 
South District guidance counselors reported that in addition to district- or school-provided 
professional development, they were free to pursue any training that supported school mandates; 
in many cases, supplemental funds were available to support such independent training. 
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Systems and Strategies to Gather Quantitative and Qualitative Data on Both POS Components 
and Student Outcomes Are Crucial for Ongoing Efforts to Develop and Implement POS 

State Assessments 

During our study, the state was moving from high-stakes tests taken in the 10th grade to end-of- 
tenn assessments. Some subject areas were piloting these end-of-tenn tests, thus creating an 
extra-long “testing season.” One counselor told us that testing lasted two months straight, 
including state tests, AP tests, and end-of-tenn tests. This change in testing did not affect our 
cohort, however, as our students had already taken their high-stakes exams. The state tests also 
went online during our study. New schools and those with updated computers were better 
resourced to manage the increase in demand created by the online tests. But because all students 
needed to take exams, students who needed to use the computers in their regular classes were 
displaced during testing periods. Insufficient numbers of computers were a statewide issue. 

A principal said that one way to increase his test scores was to dis-enroll chronically absent 
students — those not seen for months — prior to the administration of the state tests. 

South District fostered teacher professional learning communities in which teachers from the 
same subject area met to create common assessments, examine results, and detennine the best 
practices that were used to obtain those results. 

Content Standards Define What Students Are Expected to Know and Be Able to Do to 
Enter and Advance in College and/or Their Careers 

Of the three districts participating in the study, South was the only district in which participants 
offered clear-cut definitions of college and career readiness derived from state and district 
requirements — in this case, math and reading test scores for college readiness, and CTE or POS 
completer status and acquisition of an industry certification for career readiness. As a counselor 
at Jade noted, such rules were apt to change from year to year, causing confusion when 
registering students for dual enrollment, counseling them regarding their eligibility for state 
scholarships, or reviewing graduation requirements. 

Most participants, particularly at the unabashedly college-preparatory Bronze, said that college 
readiness was a state and district mandate. Other participants, like Scarlet’s principal, esteemed 
job readiness as a goal of equal worth, stressing that the school’s focus was to graduate students 
with the diplomas, skills, and certifications needed to get a well-paying job, whether they 
attended college at the same time or not. As a counselor at Lavender noted, many students “will 
go on to the state college. . . I do appreciate dual enrollment and its ability to get their feet on that 
campus during high school and see that it’s not a scary place, and to save a little money.” As she 
explained, the state college offered programs and training that allowed students to “have a better 
lifestyle for themselves and their family.” The academy coordinator at Plum agreed: 

We prepare them to leave here and go on to whatever the next step of [their lives are] and 
we do push college. I know it’s not the perfect thing for everyone, but education is the 
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key, whether it be college or whatever it is. You’ve got to be trained in something to be 
able to go out there and make more money and have the better job and be the person 
that’s going to be hired. . . That’s what we believe. 

Guidance Counseling and Academic Advisement Help Students to 
Make Informed Decisions About Which POS to Pursue 

Communicating the range of college and career preparatory opportunities available to students 
and parents required the joint participation, knowledge, and experience of guidance counselors, 
academy coordinators, and highly skilled CTE teachers with strong industry backgrounds. 

Students’ electronic IGPs provided a structure around which many guidance and career 
counseling activities took place. These plans outlined all of the possible permutations of courses 
available within students’ pathways, as well as available dual credit and online courses, options 
for postsecondary study, and career choices. 

Of course, students choose POS and other high school programs for reasons beyond career 
preparation, and school staff recognized this. Two principals we interviewed noted that some 
students needed a “hook” in order to keep coming to school. For some it was athletics, for others 
it was band or art; and for others, it was POS. Periwi nk le’s principal put it this way: “You have 
to find something these kids enjoy coming to school for. You know they’re not coming here to 
enjoy calculus and they’re not coming here to enjoy chemistry. But if you can tie it to medicine, 
which is something they have a desire for, they’re going to excel, and that’s the important part of 
having these type of programs.” Scarlet’s principal concurred: 

Every one of those students that goes through the construction program for the most part 
really buys into it. They enjoy the program, and then the same thing for the other 
programs - for the IT or for the biotech - they enjoy it. It doesn’t mean that every single 
one of them are going into a biotech career when they are done, but they are focused 
students and it gives them something that they enjoy coming to school for, and that’s the 
number one reason for any of these programs is to have that hook that makes them want 
to come. 

Guidance Counselors ’ Roles and Responsibilities 

Schools differed in how they distributed counselors’ student caseloads — some dedicated 
counselors to specific POS, whereas others employed school-wide alphabetical caseloads. In the 
latter case, schools used academy coordinators as liaisons between CTE programs and the 
guidance department. Bronze took a different approach. There, deans (essentially assistant 
principals) served as counselors, handling scheduling, course registration, and testing; a 
dedicated college counselor offered college preparatory services like application reviews and 
scholarship and financial aid assistance. 

Counselors’ responsibilities looked the same regardless of the structure (academy-specific or 
alphabetic) of their heavy caseloads. Counselors oversaw school scheduling, conducted annual 
IGP updates, registered students for courses, performed graduation credit checks and transcript 
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analyses (including to detennine whether students could qualify as POS completers), 
disseminated college and financial aid information, coordinated campus visits and college fairs, 
offered sessions on credit transfer opportunities, and administered state high-stakes tests and AP 
exams. In some schools, guidance counselors were responsible for overseeing virtual high school 
and/or online coursework and credit recovery programs; in others, academy coordinators 
undertook this task. Counselors also reported spending some time managing students’ social and 
emotional problems, although a counselor at Periwinkle believed that counselors needed more 
support regarding student mental health and substance abuse issues. Most counselors reported 
that they did limited career guidance and advising; most promoted the use of the state’s careers 
website and administered interest inventories and assessments like the ASVAB. Career guidance 
was largely the purview of academy coordinators and teachers. 

Academy Coordinators and Other Support Personnel 

At Scarlet, the academy coordinator said that in order to be college and career ready, her students 
had to graduate with something other than just a diploma — they should leave school with the 
knowledge, credentials, workplace skills, and college credits required to thrive in a range of 
postsecondary and occupational settings. For that to happen, students needed to engage in 
enriched classroom instruction, participate in work-based learning and credit transfer 
opportunities, and receive excellent career guidance and counseling. Academy coordinators were 
the drivers of many of these activities. 

Depending on the school, academy coordinators had multiple responsibilities, including, but not 
limited to, facilitating the district’s lotterized academy application process, serving as liaisons 
between the school’s academies and the administration, collecting academy accountability data, 
overseeing middle school outreach and recruiting, administering industry-recognized credential 
exams, registering students for dual enrollment, soliciting guest speakers, coordinating special 
events, scheduling work-based learning opportunities, and maintaining relationships with 
business and community partners. Plum’s academy coordinator described herself as part teacher, 
part counselor, although she carried a reduced caseload of IEP students compared to the regular 
guidance counselors’ caseloads of general population students. 

Career Guidance and Career Exploration 

Career guidance and career exploration were embedded in academy curricula and co-curricular 
activities for all of South’s career academies. The goal of this career guidance was to help 
students understand the full range of college and career options available to them. Academy 
teachers, many of whom came from business and industry, enriched their classrooms with hands- 
on learning experiences that brought students into contact with business and community partners 
while they engaged in work-based learning and earned industry-recognized credentials. The 
supervisor of South’s medical academies said that the district had invested heavily in 
establishing legal agreements with area hospitals, clinics, nursing homes, and other facilities in 
order to make such hands-on training possible. 

The district’s medical and health science academies offered an excellent case study of the types 
of career guidance and exploration available to students through in- and out-of-class experiences, 
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including in-class clinical practice, internships, off-campus clinical placements, and CTSO 
competitions. At Lavender, we observed a health science class taught by a registered nurse in 
which students practiced nursing skills in preparation for the CNA exam. Lavender was an 
approved CNA testing site, and this classroom included a fully equipped medical bay featuring 
hospital gurneys, medical dummies, feeding stations, and supplies. The teacher explained how 
her students divided their time between the classroom and the lab, then applied and practiced 
their skills in a series of clinical placements at area nursing homes and medical facilities. 

Students kept journals in which they logged their clinical experiences. The teacher explained 
how these journals not only taught students how to write proper case notes, but also encouraged 
them to engage in critical self-reflection — a valuable skill in the high-stress world of healthcare. 

The students we interviewed spoke enthusiastically of their clinical experiences and saw them as 
sources of skill development and career exploration. A medical academy student at Plum 
mentioned rotations in the histology and hematology labs at a local hospital, a nursing home 
placement, and an EMT ride-along. These field experiences, combined with his teachers’ advice, 
input from visiting guests, and HOSA leadership activities, led him to identify pathology as a 
potential career. A Lavender medical academy student made a similar statement, noting that her 
internships had helped her gain skills and an understanding of what she wanted to do after 
graduation. Coupled with the valuable industry-recognized credentials she had earned, these 
experiences would allow her to get jobs that would help her pay her way through medical school 
at the in-state university to which she had applied. 

Teachers also found creative and challenging ways to blend career guidance into their 
instruction. At Periwinkle, the medical law and ethics teacher — a former nurse and expert 
witness — explained how her students explored real-world legal issues facing doctors and medical 
professionals by thoroughly researching local court cases and conducting in-class trials. This 
teacher sought to prepare students not only for the rigors of medical training but also for the 
ramifications of their career choices. In a long-term project, her students researched potential 
careers and reported on associated colleges or training programs, curricula, study requirements, 
tuition and financial aid, and lifestyle costs (cost of living vs. earnings). 

Many of the students we interviewed spoke in glowing terms of the influence, expertise, and 
“amazing help” offered by their CTE teachers, describing them as role models and even friends. 
A Periwinkle medical academy student said that her teachers — “real doctors and nurses” — had 
helped her understand the opportunities available to her in health care and had been instrumental 
in helping her narrow her college applications to the programs and schools that would best suit 
her interests. A Lavender sports medicine student believed that familial relationships and strong 
bonds between teachers and students — nurtured through co-curriculars like HOSA and after- 
school athletic events — had supported her through her challenging program. 

The cohort model followed by many POS may have fostered the familial environment this 
student and others described. Medical and health sciences POS students at Bronze, Periwinkle, 
and Lavender all cited other students as a valuable source of understanding, motivation, and 
support. A Periwinkle student said that the culture of her academy encouraged student -to-student 
mentoring; upperclassmen were expected to guide, teach, and support lowerclassmen through the 
challenging early years of her program, which she likened to medical school in their intensity. 
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Graduates of these programs often returned and gave advice to current students, telling them to 
leam the subjects now in high school, and they would have an easier time in college than many 
of the others that these POS graduates watched struggle. 

Students also received career guidance and support from outside experts. A Lavender sports 
medicine student commented that guests frequently visited her health sciences POS to talk about 
their careers; such guests included an HIV/AIDS prevention expert, a professional lifeguard, 
hospital personnel, EMTs, firefighters, and sports medicine doctors. This student believed that 
these contacts were helping her make more informed college and career decisions. Although not 
all POS shared as comprehensive a career guidance and exploration approach as the medical and 
health sciences POS, all district POS attempted to provide similar career guidance experiences. 

Innovative and Creative Instructional Approaches Enable Teachers to 
Integrate Academic and Technical Instruction and Students to 
Apply Academic and Technical Learning in Their POS Coursework 

Interdisciplinary Teaching Teams 

South District embraced the idea of rigor and relevance in the high school curriculum and saw 
POS as a means of achieving both. Most POS cohorted their students, with English, science, and 
social studies teachers dedicated to each program. Program teachers planned POS-specific cross- 
curricular instruction during their only common planning time, the high schools’ half-dozen 
regularly scheduled morning in-services. Few high schools could institute any other common 
planning time. The state’s class size reduction law for core academic classes made cohorting and 
common planning time much more challenging to maintain for those schools that could offer 
them. For example, under the law, if a school wanted to cohort all 30 of its construction POS 
students in the same English class, only 25 could actually be in the class; the remaining five 
would be separated from their cohort. 

We heard of geometry teachers working with construction teachers, history and English teachers 
working with a media program, and a culinary teacher working with a drama teacher to create a 
murder mystery dinner theater event. A medical POS teacher said that he “camouflaged” the 
state academic standards in his program; he also brought in academic teachers to help teach 
embedded concepts. Administrators who reviewed his program told us they were unable to tell 
“who was the medical and who was the English teacher.” 

During our classroom observations, we did see academic classes tailored to a cohort of POS 
students, as was the case with a medical POS chemistry class at Plum. However, we observed 
more instances of POS teachers incorporating academic subjects into their classes, including an 
auto body class that reviewed optimum paint/air ratios and air pressure readings for the various 
paint guns they used, as well as culinary students taking a quiz that looked more like a science 
quiz (“Which of the following is used to measure humidity levels in dry-storage areas?”). We 
saw no teacher teaming in our observations at South District. 

Student perspective. The students we interviewed provided many examples of how academic 
concepts were taught in their POS classes. One medical careers student said, “Oh, all the time. I 
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mean medical, it’s from math, science, a lot of reading, a lot of history — the history behind 
medical, you know, how medicine came to where it is. The math, always adding numbers and 
how many cc’s of this and just adding, in kilograms, in grams, in weight and measurements. 
Science, just basic body science, you know.” 

Students also recognized when their academic teachers made connections between their subject 
area and the world of work: 

All of my teachers-, they’re like life lessons in their lesson plans kind of. Like my math 
teacher, she’ll just kind of go off on how you can use what we’re learning in class in the 
real world. And also my chemistry teacher, he’ll do the same thing. You know, when he’s 
working in the labs, how he was applying it, and it kind of helps me to see how you can 
actually use the stuff we’re learning, cause sometimes it doesn’t make sense to me. It’s 
like, when are we ever going to have to use this? But then they’re like, “This is when 
you’re going to use this.” and it also depends on what career you’re going into but this is 
just good general knowledge and it helps me see what I want to do. 

Students noted differences in the concerns of their academic and POS teachers, as described by a 
student in the medical careers program: “I know our medical teachers, they’re always asking 
what colleges did I apply to, you know, am I getting my stuff done like scholarships and things 
like that. Usually regular teachers, they won’t really get on you, like ‘what are you doing to go to 
college?’ and stuff like that. I think that’s one of the reasons it’s good to be in a career academy, 
because the teachers, since their focus is only on you, they’ll actually try to understand what 
you’re going to do in the future and try to help you out.” 

Contextualized Learning Approaches: Work-Based Learning 

South District’s commitment to providing work-based learning opportunities to every CTE 
student was apparent at every school we visited. Although it took time, significant resources, and 
industry cooperation to secure and maintain internship and clinical sites, district and school 
personnel saw work-based learning as a means of preparing students for life after high school, 
whether students chose immediate employment, enrollment in a college or technical school, or a 
mixture of both. As Scarlet’s principal said, their goal was to show students what was possible, 
potentially saving them both time and money: 

What we’re doing is giving them alternate paths that they may be interested in and giving 
them a taste of it. . . letting them see what it’s like to work on a car all the time or to build 
a house. To let them know if they’re gonna like it or not, before they get out of here, so 
that they don’t have to leave high school with a diploma and no direction, and then. . . go 
sample jobs for several years until they decide what they want to do. 

Work-based learning allowed students to explore and apply what they had learned in theory and 
by so doing, to not only identify their career goals, but also tap into their personal passions. As 
an automotive student at Plum said, “My ultimate goal is to own my own body shop. [When] I 
first came in, I didn’t want to do body work, but I came in and I started getting around it and I 
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just kind of fell in love with it. . . it was the hands-on — it’s. . . a rush when something rolls out. 

It’s like, ‘I did that.’” 

Work-based learning in South District frequently took the fonn of off-campus internships and 
clinical placements, but also included school-based enterprises and other on-campus activities. 
Teachers and students described a broad and interesting range of internships, including an 
environmental studies student conducting site surveys for an engineering company, broadcasting 
students taking summer jobs with a local Clear Channel affiliate, culinary students earning hours 
toward their certification by working for local restaurants, and drafting students designing 
improvements to a community skate park. 

As we have described elsewhere, no program placed a greater emphasis on work-based learning 
than South’s medical and health science programs, which required students to complete 
internships. The district’s coordinator for these programs maintained some 100 active contracts 
with various area medical facilities, including hospitals, clinics, physical and sports therapy 
providers, labs, nursing homes, ophthalmology clinics, morgues, and pharmacies, among others. 
Lavender CNA students practiced their skills by bathing and feeding residents and changing beds 
at area nursing homes, then reflecting on those experiences in field journals. A Plum health 
sciences student who was indecisive about his future career relished the fact that his program 
offered many internship and job shadowing opportunities that had allowed him to explore the 
real world of pathology, histology, and emergency medicine. 

Some schools featured school-based enterprises that went beyond simulating real workplaces by 
offering actual services to the public, as was the case for Lavender’s early child development 
program, which housed an on-campus preschool. One of Lavender’s school counselors had a 
grandchild enrolled in the preschool, which received over 500 applications for 20 available slots. 
Students gained 480 hours of work experience in preparation for certification. As the counselor 
noted, both the program and the preschool allowed students to explore the full range of career 
options in the field. 

Where off-campus internships or school-based enterprises were not available, programs sought 
to provide robust in-school work-based learning and community service experiences. Although 
she had also participated in formal internships, a Lavender sports medicine student completed 
more than 2,000 hours of sideline sports medicine at school events. Lavender IT students 
repaired and rebuilt computers for needy families and earned community service credits at the 
same time. Plum medical academy students hosted community health fairs at which they took 
blood pressure readings and earned scholarship money for each blood donation attendees gave. 

Contextualized Learning Approaches: Project-Based Learning 

Although project-based learning did not appear to be a mandated district initiative, we saw 
evidence of implementation of this approach at each of the schools we visited. The depth and 
intensity of project-based learning varied by program, but frequently included examples of 
curriculum integration — as with culinary students making ratatouille, cream puffs, and baguettes 
in conjunction with a history unit on the French Revolution, or a public service announcement 
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film project that was tied in to students’ English classes. When available, common planning 
periods, or even informal teacher planning at lunch, greatly facilitated project-based learning. 

Some of the best examples allowed students to solve real-world problems in their communities 
and also drew upon the strengths and contributions of schools’ community and business partners. 
As noted, Scarlet’s construction technology and design and drafting students worked with area 
business partners — including professionals associated with the school’s ACE Mentoring 
chapter — to design and build a Habitat for Humanity home. The district negotiated an extended 
contract with Habitat for Humanity that would allow students to complete construction of the 
home over a longer period of time than other Habitat homes and also purchased insurance to 
cover student workers. Community members and business partners donated their time and 
expertise, building materials, and the land on which the house was to stand. 

Similarly, juniors and seniors in Lavender’s design, engineering, and construction programs 
undertook the redesign, rehabilitation, and reconstruction of a historic neighborhood property 
known as Lavender House. This example of project-based learning was not without its 
problems — the program’s instructor commented on the high cost of construction pennits, the 
difficulty of making changes to a landmark property, and constraints on what students could 
accomplish at the site given the school’s schedule. However, two seniors talked about how the 
project not only allowed them to earn community service hours and apply their “book 
knowledge” of design and drafting to a real building, but also revealed their work styles and 
career preferences. One student talked about how much he loved both the mechanical world and 
the hands-on aspect of construction; Lavender House allowed him to plan, build, and experience 
all aspects of the process. The other student said that although he enjoyed seeing his designs 
translated from concept to reality, he discovered that he preferred designing and supervising to 
hands-on construction work. However, he recognized the value of practical experience: As he 
said, the program gave him “a lot of book knowledge. . . But the thing is, if . . . we’re designing 
something and then we come to figure out that later on it doesn’t fit properly in the actual plan, 
that could cause a lot of conflicts. . . so it’s good that we have that [experience] also.” 

Teaching Soft Skills 

All high schools in South District had two types of POS: those accessed through the lottery and 
those for students in their zoned high schools. The latter often did not begin until sophomore or 
junior year, although some prerequisites (e.g., biology for medical POS) were needed prior to 
joining the POS. At Lavender, the construction POS was for local students only. The teacher 
required that introductory level courses be taken prior to joining the program; during these 
introductory courses, he decided whether students were trustworthy enough to join it. Maturity 
was more important to this teacher than initial capability. 

South District students received important soft skills instruction during internships, which were 
strongly encouraged but not required in most POS. Internships usually offered highly structured 
opportunities for employers to teach soft skills. Employers evaluated students’ performance and 
eligibility for high school credit on assignments requiring information-seeking and networking, 
among other activities. Many students identified their own internship locations; one student 
interned at his father’s workplace, where he developed an inventory spreadsheet for the company 
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warehouse using “first- in, first-out” principles. Such experiences helped students develop the 
teamwork and problem-solving skills employers desired. 

Team-building. Over the course of our site visit to South District, we observed fifteen POS 
classes and three academic classes. Although we saw no examples of teamwork in the academic 
classes, POS classes offered many examples, some more challenging than others. Students 
worked together to organize and file patient identification records according to a tenninal digit 
numerical filing system, baked pumpkin bread, taped each other’s ankles, and edited film footage 
together. In a construction class, students worked in crews identified by different-colored hard 
hats. Each crew had a supervisor and a quality control person, among other roles. Role 
assignments changed over time, giving students experience working in different contexts. For 
example, students acting as team supervisors had to identify and train their own replacements. 
Students assigned the task of quality control could stop the work at any time if they saw 
something go amiss. In one such instance that we observed, quality control flagged a problem 
with the frame of Scarlet’s Habitat House; students paused, considered the issue, and negotiated 
a solution by prying out a number of nails and starting over. 

Critical thinking and problem-solving skills. Classroom observations offered many examples of 
students applying previously learned information to the solution of problems; in some classes, 
students synthesized information from a variety of sources. In a biotechnology class, for 
example, students completed a lab assignment on the effects of chemical buffers (e.g., alkaline 
compounds) on acidic solutions, then moved to the classroom’s computers to visually represent 
their results in Excel tables and graphs. The assignment required students to apply, synthesize, 
test, and communicate both their biotechnology knowledge and their computer skills. 

A Law and Medical Ethics class featuring preparations for a mock trial provided a good example 
of students thinking critically and generating meaning to arrive at a “real” outcome. Students 
synthesized legal and medical knowledge as they developed prosecution and defense strategies in 
a murder case. We watched students work together and with their teacher to compose questions 
and opening statements, understand mitigating circumstances, present evidence, and understand 
such concepts as blood alcohol content, rates of metabolization, and the effects of head injuries. 
Students also practiced typical academic skills like effective composition and communication. 
Although this was not a real case, students appeared earnest in their efforts to apply their medical 
knowledge to upholding the law. This teacher sometimes used real closed cases in her class. 

Teaching soft skills through CTSO activities. CTSOs were popular and well supported in South 
District. However, staff at some schools told us that competitions were so demanding that some 
teachers could not make the time commitment needed to support them. One district administrator 
said: “I support [CTSO participation] but I won’t stop talking to teachers if they decide not to do 
it. Their plate is very full.” One principal said that because many POS classes increased in size 
after the passage of the state’s class size reduction law, many teachers found it difficult or 
impossible to continue to sponsor CTSOs. 

We met a number of students in CTSO leadership roles; these seniors were among the most 
directed, purposeful students we interviewed. One student, an FBLA president, said he initially 
joined only to boost his resume, but found the experience to be much more than he imagined. He 
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was one of several CTSO leaders we spoke with who were taking AP and dual enrollment 
classes, earning industry-recognized certifications, and accepting early admissions offers from 
the colleges of their choice. 

One Bronze student joined the school’s medical academy after learning about and joining the 
school’s chapter of HOSA, the health sciences CTSO. HOSA participation sparked her interest 
in a medical career. She became her chapter’s president and a regional officer and participated in 
many leadership activities. She also told us she was considering a state leadership role. She was 
in charge of planning regional competitions and a health fair at Bronze; a dozen high schools and 
hundreds of students planned to attend. Like many students we spoke with, this student initially 
had not planned to become an officer, but became more involved in leadership roles as she 
participated in more regional and state competitions and helped her peers stage successful events. 

Every school we visited boasted of their students’ success in regional, state, and national CTSO 
competitions. A medical POS student we interviewed described one such competition in which 
students received a list of possible medical-related activities they could be asked to do and what 
they would have to do to compete: “You have to literally to be your own leader and carry 
yourself through whatever they throw at you.” In addition to teaching soft skills through 
competitions, CTSOs fostered mastery of these skills through hands-on community events like 
blood drives and health fairs. 

National, State, and/or Local Assessments Provide Ongoing Information on the Extent to 
Which Students Are Attaining the Necessary Knowledge and Skills for Entry Into and 
Advancement in Postsecondary Education and Careers in Their Chosen POS 

Technical Skills Assessments 

This state used industry-recognized credentials as their technical skills assessments and provided 
a list of approved credentials for most program areas. For those programs lacking an industry 
credential, districts could choose their own certifications. South District assumed the costs of 
these credentialing exams for students. Some credential exam scores were not reported back to 
the schools; students received their scores individually. In order to secure these scores for school 
reporting purposes, the district added a cover page to them that required student and counselor 
signatures. To improve student outcomes on more difficult credentialing exams, South District 
cultivated middle school pre-academies that started students sooner on the path to preparing for 
these exams. 

Some program areas, like IT, had a certification at the end of every class: One student we spoke 
with was certified in Word, Excel, PowerPoint, Final Cut Pro, and Photoshop. Other programs, 
like automotive, had only one certification as a culminating exam. Some credentialing exams 
could be taken more than once; others, due to their cost or other restrictions, could not. 

Initially, the state’s goal was to introduce these certifications to students without the expectation 
of high pass rates. They incentivized schools to offer industry-recognized credentials by 
awarding them points toward their school grade for student participation in courses with such 
credentials. Many study participants reported that principals were eager to have a credential in 
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every program in order to boost their school grades. Industry-recognized credentials thus 
increasingly played a greater role in the school grading system. One counselor told us that 
although this push created pressure for students to succeed, she appreciated the fact that CTE 
was being recognized as a significant part of school accountability: “If our school is going to be 
successful, our CTE classes need to be successful.” 

Passing an industry credential exam represented meeting one element of the district’s definition 
of career readiness. Consequences for not passing varied. In some cases, we were told that failing 
an exam did not affect a student’s classroom grade; in other cases, it did. Failing an exam did not 
affect graduation status. Participants did say that failing to pass the credentialing exam affected 
students’ prospects. A counselor said that industry credentials — and the well-paying jobs they 
could help students secure immediately after graduation — were a big selling point when 
recruiting middle school students. Both students and school personnel acknowledged that 
credentials held value as a means of earning money to help pay for college, even if students did 
not plan to pursue careers in the same area. For example, although South’s medical programs 
were not strictly pre-med, many students in those programs planned to become doctors. The 
certifications they earned in their programs provided them with jobs, experiences, and exposure 
to different medical fields and potential careers much earlier than students who did not 
participate in such programs. 

Assessments and Standards in the Classroom 

Unlike students in West or East District, South District automotive students were encouraged to 
take and pass the ASE certification exam. Preparing for this difficult credentialing exam required 
more classroom hours than most high schools were able to provide, but given the state’s use of 
industry-recognized credentials as technical skills assessments, as well as the district’s heavy 
focus on them, South students were able to accumulate the coursework hours needed to take this 
exam. Students still needed to acquire work experience hours in order to receive the ASE 
certification and a patch for their work uniforms. One proud student who successfully passed his 
ASE exam said this patch would announce that “I know what I’m doing.” 

As noted, many industry exams were taken online, which caused problems for schools with 
inadequate numbers of computers or outdated equipment and software. Participants mentioned 
dropped internet connections and lost work; teachers said they were unable to give their students 
administrator passwords so they could practice for exams that would allow them to become 
system administrators. Although this specific issue was legal, many technical issues were 
attributable to budget cuts. A focus group of IT students told us that in one instance, they could 
not take practice exams until the schooFs site license was renewed. Once it was, the new 
software conflicted with the school’s old hardware, which had not been updated due a lack of 
funds. Having practiced on the old software, students were shocked to discover during the timed 
final exam that the new software used different function keys. They also complained of having to 
take some exams at a regional testing center, the unfamiliar setting of which added to their stress. 
Students were quite aware of the stakes involved: “The main focus of the academy is for the 
testing... because that’s how we’re measured.” 
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CHAPTER SIX: Telling the Story of POS 
Cross-District Synthesis 

In this chapter we integrate outcomes from this mixed-methods study to synthesize what we 
know about the impact of POS on student achievement. Results from all three districts are 
presented here and integrated with qualitative outcomes for the four mandated elements of POS. 

Cross-District Outcomes Synthesis 

As noted in Method, we operationalized the four mandated elements of Perkins IV into measures 
that capture the skills and content that POS were designed to address. Most of the academic and 
technical achievement expectations of POS are found in the second element of the law; transition 
outcomes are found in the last two elements. We did not quantify the first element (i.e., the 
extent of secondary and postsecondary alignment). Instead, qualitative data were used to 
synthesize all findings in that regard. We present the four elements again in Table 6.1. 

Table 6.1 


The four mandated elements of POS 


i 

POS must incorporate secondary education and postsecondary education elements. 

ii 

POS must include coherent and rigorous content aligned with challenging academic 
standards and relevant career and technical content in a coordinated, nonduplicative 
progression of courses that align secondary education with postsecondary education to 
adequately prepare students to succeed in postsecondary education. 

iii 

POS may include the opportunity for secondary education students to participate in dual or 
concurrent enrollment programs or other ways to acquire postsecondary education credits. 

iv 

POS must lead to an industry-recognized credential or certificate at the postsecondary level, 
or an associate or baccalaureate degree. 


Source : Perkins IV, Section 122[c] [1][A] . 


Incorporate and Align Secondary and Postsecondary Education Elements 

In all three participating districts, POS incorporated elements of secondary and postsecondary 
education, primarily in the fonn of credit transfer agreements, which we describe below in 
Postsecondary Credits. In this section, we discuss other ways in which the districts aligned 
secondary education with their postsecondary partners. 

CTE curricula in all three districts were state-approved and vertically aligned from secondary to 
postsecondary education. Postsecondary faculty members participated in high school CTE 
curriculum development in order to ensure that students would be able to transition from one 
system to the other without duplicating courses or content. Across districts, what differentiated 
POS from regular CTE programs was that students participating in POS took courses in 
structured, progressive sequences culminating in a capstone course and they benefited from the 
other elements of POS, such as intensive course experiences, integrated academic and career 
curricula, the expectation that students would attend college, and sustained career development 
and guidance. In contrast, regular CTE students could take any combination of CTE courses 
offered at their high school, within or across programs, with or without a sequence or a 
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culminating capstone course. Regular CTE did not necessarily offer college credit or lead 
seamlessly to articulated, college-level programs. 

Alignment between secondary and postsecondary education appeared to be most advanced in 
West District, where in addition to curricula aligned with postsecondary programs, many West 
POS also aligned textbooks, faculty, and facilities across institutions. In East District, a similar 
level of close alignment once existed between the programs offered by the district’s 
postsecondary partner, ECC, and those at Blue Academy. The recession weakened these ties. 
Creating aligned POS came at a cost to postsecondary partners. Community college 
administrators in both West and East were concerned about the return on their investment of the 
faculty time, facilities, and other resources needed to sustain these partnerships. 

In both East District and South District, interviewees said that curricular alignment largely 
occurred at the state and district levels as opposed to the school level. From our interviews, it 
appeared that faculty had more opportunities to interact across institutional levels in West 
District than in East. We did not interview teachers during our one site visit to South District and 
thus know less about cross-institutional collaboration there. Student outcomes related to these 
alignments are discussed in Postsecondary Credits, below. 

Include Academic and CTE Content in a Coordinated, 

N on-Duplicative Progression of Courses 

In general, all three districts developed aligned, non-duplicative course sequences in conjunction 
with postsecondary partners, predominantly but not exclusively at the local community college. 
Sequences included introductory, mid-level, and advanced or capstone courses and opportunities 
for students to earn college credits, as established by state- or district-negotiated articulation 
agreements. Districts varied in how they created, implemented, and maintained these sequences. 

Working with significant input from its secondary, postsecondary, and industry partners, West 
District took a backward-mapping approach to designing and implementing non-duplicative 
course sequences by beginning with where it wanted students to end up, then creating pathways 
to get them there. Based on the contexts and needs of specific schools and programs, West 
offered sequences that could be completed in three to four years of high school coursework 
before leading to college programs. West’s sequences were designed to offer opportunities to 
earn college credit and industry-recognized credentials, where appropriate. 

As our study began, program requirements for POS completion in East District mandated that 
students follow tightly-defined four-course sequences. Later changes made to the state’s high 
school graduation requirements compelled East to alter its approach to creating course 
sequences. Although the new graduation requirements did not affect our cohort, the ongoing 
recession did force many schools, including Blue Academy, to close programs and cut upper- 
level courses with low enrollments. Such closures and cuts had a negative impact on students in 
our cohort, many of whom had to take classes outside their tightly defined sequences. As a 
result, many students in our cohort failed to qualify as POS completers. 
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POS sequences in South District’s career academies were comprised of four progressive 
courses; these sequences were also vertically aligned with feeder middle school programs, SCC, 
and other in-state postsecondary institutions. The district envisioned these sequences as 
“roadmaps” that included industry-recognized credentials, work-based learning experiences, and 
opportunities to earn college credits. 

Academic and Technical Achievement Outcomes 

Most of the quantitative outcomes in the study fit under this POS element. These outcomes 
consist of the two measures from the instrumental variable analysis (graduation and overall 
GPA) and four measures from the posthoc analyses (overall GPA, STEM credits earned, AP 
credits earned, and GTE GPA). 

Graduation. The results of this study suggest that (1) West District students who enrolled in 
POS schools were about 1 1% more likely to graduate than students enrolled elsewhere and (2) 
this benefit was attributable to their increased earning of GTE credits. Our study of West District 
suggests that earning GTE credits translated enrollment at a POS school into graduation. That is, 
earning GTE credits mediated the effect of enrollment on this outcome. 

Students in East District who earned more GTE credits were also more likely to graduate. 
However, the results for East did not support the type of causal inferences we were able to draw 
from the West District data. This was because enrollment at the POS school did not perfonn well 
as an instrument for estimating the effects of earning GTE credits. Enrollment in the POS school 
only increased the number of GTE credits earned by an average of .20, or by less than one 
additional GTE course. Thus, the instrument generated very little exogenous variation in number 
of CTE credits. Several characteristics of East District may explain why enrollment at Blue was 
not a good instrument. First, East had strong CTE offerings district-wide, and many students 
participated. Second, POS completers and CTE concentrators were both required to earn four 
CTE credits. This perhaps contributed to the failure of enrollment at Blue to contribute to earning 
more CTE credits. Finally, the recession and the resulting reductions in capstone courses that 
occurred at Blue meant that some students failed to complete their POS, further diminishing the 
difference between POS students and other CTE students in the district. 

At South District, enrollment in POS did not generate sufficient variation in the number of CTE 
credits earned and thus was not a good instrument. At South, students could enroll in POS at 
their zoned high schools without going through a lottery, so many students in the comparison 
group also earned similar numbers of CTE credits. Intervention students who enrolled in South 
District POS were about 9% more likely to graduate than students who enrolled elsewhere. In 
addition, earning more CTE credits was associated with a greater probability of graduation. 
However, the causal inference at South must remain tentative due to the poor performance of the 
instrument. 

In sum, enrolling in a POS in ninth grade and taking the additional CTE credits associated with 
that enrollment significantly improved the likelihood of graduation in West District. In East and 
South, such strong causal claims cannot be made; however, earning increased numbers of CTE 
credits was associated with graduating in these two districts and enrollment in POS was 
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associated with improved likelihood of graduation in South. This is consistent with prior work 
showing that CTE is a positive contributor to students’ chances of graduating from high school 
(Aliaga, Kotamraju, & Stone, 2014; Plank, 2002; Plank, DeLuca, & Estacion, 2005; Stone & 
Aliaga, 2003). 

Overall GPA. The instrumental variable analysis in West District showed no difference between 
the intervention and comparison groups on overall GPA, but the posthoc analysis of POS 
completers showed that they had a higher GPA than either the CTE concentrators or the rest of 
the sample. The strong effect sizes indicate that completing a POS at West had substantive 
positive effects on a student’s GPA. In addition, we observed a school effect there, with students 
attending Navy being significantly more likely to have a higher overall GPA than students 
attending Azure or Sky. This could be attributable to Navy’s structure, curriculum, and culture, 
which may have engaged students and shown them how achievement in both academic and 
technical courses was necessary for the careers they wished to pursue. 

Intervention students in East District who earned more CTE credits were more likely to have 
higher GPAs. But as with our graduation analysis, the results for East did not support the type of 
causal inferences we were able to draw from the West District data, due to the poor perfonnance 
of enrollment at the POS school as an instrument for estimating the effects of earning CTE 
credits. We found no differences between the East posthoc groups in overall GPA, meaning that 
students were able to complete POS at no negative cost to overall grades. 

Our findings for South District were similar to those for East: Enrollment in POS did not 
generate significant variation in the number of CTE credits earned and thus was not a good 
instrument. Intervention students who enrolled in South District POS were more likely to have 
slightly higher GPAs than students who enrolled elsewhere. In addition, earning more CTE 
credits was associated with a greater probability of earning a slightly higher GPA. However, as 
with the graduation outcome, the causal inference on GPA at South must remain tentative. 

AP and STEM credits earned. Completing a POS made no difference on earning AP credits in 
West District. Because West’s sample consisted of all students who applied to the three POS 
high schools, this result suggests that this sample of students would have earned similar numbers 
of AP credits regardless of the POS intervention. 

However, STEM credit earning in West District shows differences among our posthoc groups. 
Perhaps due to the high-tech POS at the three intervention schools, students took POS-relevant 
math and science courses, leading to their earning more STEM credits than the rest of the sample 
or the CTE concentrators. The substantive effect size on the latter finding indicates that there was 
a difference between completing a POS and completing a CTE concentration at the comparison 
schools. One difference we observed was that in West District, some concentrations only 
required two credits. Such concentrations, such as Foods or Graphic Design, seemed less likely 
to foster extra math or science course-taking than the analogous four-credit POS sequences (i.e. 
Culinary Arts or Architecture), many of which followed a different scope and sequence despite 
similar program names. Other differences between POS and CTE concentrations in West District 
included college preparatory academics at the POS high schools, business partnerships, and 
alignment with postsecondary programs. 
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In East District, Blue Academy personnel held the expectation that all Blue students would 
attend college. We did not observe this to be the case at the comparison schools, so the higher 
AP credits earned by Blue POS completers compared to the rest of the sample is not surprising. 
The effect size of this outcome, the highest in the study, indicates that completing a POS at Blue 
may have had substantive positive effects on earning AP credits. However, when we examined 
CTE concentrators at Blue, we noted that the POS completers’ advantage in AP course-taking 
disappeared. We posit that this outcome might be a result of a school effect — Blue Academy — as 
opposed to a POS effect. In other words, the result is likely a function of the expectations and 
culture at Blue, not of completing a POS. In addition to school culture, another factor influencing 
course-taking in East District may have been the severe budget cuts that resulted from the 
recession: Many capstone courses at Blue were cut in favor of higher-enrollment courses. This 
led to two consequences related to advanced academic course-taking: First, many students were 
unable to complete their POS, making them involuntary CTE concentrators instead who did not 
otherwise differ from the POS completers except in the opportunity to complete their POS. 
Second, given the paucity of capstone courses available, students had more room in their 
schedules to take more AP courses. All of these possibilities may have contributed to these 
outcomes at East. 

Completing a POS in South District made it significantly less likely that students would earn 
either STEM or AP credits, compared to both CTE concentrators and the rest of the South 
sample. Because the rest of the sample consisted of students who applied to other district choice 
programs, it seems clear that students choosing strong college preparatory programs like IB, pre- 
law, or STEM would be more likely to earn more credits in AP or STEM coursework. 

CTE GPA. Completing a POS appeared to have a significant positive effect on CTE GPA for 
completers in West and South Districts, compared to the rest of the students in our respective 
samples, with substantive positive effects in West District and moderate effects in South District. 
In East District, there were no significant differences in CTE GPA among POS completers, 

Blue CTE concentrators, or the rest of the sample, although there was a large effect size for the 
comparison with Blue concentrators, indicating that completing a POS in East District may have 
had substantive positive effects on a student’s CTE GPA, even though it was not statistically 
different from the concentrator GPA. 

This was not a self-explanatory outcome. One can imagine that taking more CTE, and taking 
increasingly more difficult CTE courses, as POS completers did, could have meant earning a 
lower CTE GPA than other students. But where there were differences between groups, they 
always favored the POS completers, with healthy effect sizes. CTE GPA was our technical 
assessment measure, so we conclude that completing a POS improved results on assessments of 
student technical knowledge and abilities. 

Context for the Achievement Outcomes: West District 

Regular CTE programs in West District did not offer the same rigor as POS — this is evident in 
both GPA and STEM credits outcomes. POS completers had significantly, substantively higher 
GPAs than both comparison groups. That, coupled with the higher graduation rate among 
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intervention students, suggests that POS motivated those students to achieve and finish high 
school. We posit that in addition to completing more CTE credits, the additional elements of 
POS helped foster engagement and achievement. Students in POS knew that their programs had 
a built-in next step at the local community college or university. School cultures at the POS high 
schools fostered individual and career development, opportunities to lead, positive attitudes 
about achievement and greater commitment to careers. Students knew that their curriculum was 
at a higher level than at other schools — that in many cases, their preparation was college-level 
and could lead to advanced status in college. 

Students at Navy, who had a higher overall GPA than that of the POS students at either Azure or 
Sky, benefited from conditions there. First, students benefited from interdisciplinary teacher 
collaboration, fostered by grouping programs in houses so that academic teachers shared the 
same physical space as POS teachers. Navy students also benefited from the school’s project- 
based learning curriculum, which included school-wide as well as classroom-based projects. 
Navy students enjoyed authentic work-based learning experiences and school-based enterprises 
on their campus such as an operational child care facility and active culinary and hospitality 
programs that often hosted community and district events. 

Context for the Achievement Outcomes: East District 

The entire Blue Academy staff promoted college attendance as one of the primary student 
outcomes they sought. The principal claimed that he pushed students to take AP courses, 
especially those that were in any way aligned with the POS theme. We did not observe this 
overriding goal of college attendance at the comparison schools. This may account for the 
significant and substantively important positive outcome of earning AP credits at Blue compared 
to the rest of the sample. The rest of the findings were neither significant nor did they generate 
substantive effect sizes, indicating that POS may be successfully offered in districts like East at 
no cost to other academic achievement measures. 

Context for the Achievement Outcomes: South District 

The outcomes observed in South District were not like those in West and East Districts, possibly 
due to sample selection. Our sample came from a lottery sample, but it was not a random sample 
of students from the district. Nor were the lottery results a random assignment of applicants to 
programs. As noted, nearly 95% of students were accepted into one of their three choices (89% 
were accepted into their first choice program). There were not enough students in the sample to 
perfonn a PSM as was done at East, leaving a comparison group comprised of students who 
were accepted to other choice programs such as IB, STEM, or performing arts. These were not 
necessarily students who had applied to be in a POS, nor were they matched to the intervention 
students — they were a sample of convenience in the sense that South District could not provide 
an alternate sample. Unlike the other two districts that had few significant differences at baseline, 
the sample at South showed significant differences at the onset of the study in poverty and prior 
achievement. Despite controlling for these and other variables in the subsequent analyses, 
baseline differences suggest that there might be other unobserved differences between these 
groups that could have led to the outcomes. 
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We attempted to account for school effects through (1) the clustering of students in schools in the 
instrumental variable portion of the analysis, and (2) a dummy variable in West and a single 
intervention school in East for the posthoc analyses. However, at South, intervention students 
were in a large number of high schools, sometimes in the same school that non-lotterized POS 
students and general comparison students attended. This could have confounded the results. 

CTE concentrators in South District tended to outperform POS completers on several measures, 
although effect sizes were not large. Because both concentrators and the rest of the sample 
earned significantly higher numbers of STEM and AP credits than did POS completers, it is 
possible that the additional CTE credit and other requirements of POS at South District (e.g., 
internships, industry credentials) took enough time away from POS completers that they were 
unable to pursue as many advanced academic courses. South POS completers had significantly 
higher CTE GPAs than CTE concentrators and the rest of the sample, suggesting that they had 
focused on their POS areas over advanced academic course-taking. 

The STEM and AP outcomes suggest that South’s comparison group included many high 
achievers. We know from the qualitative data that there were many college preparatory programs 
in the comparison group. AP credits should be the most comparable measure across districts, 
because they have a common curriculum and exams, regardless of where classes are taught. The 
results reviewed in each chapter show that South District comparison students took almost twice 
as many AP credits as any other group in the study, in any district. This reflects the advanced 
academic standing of this comparison group and helps explain the lack of positive outcomes for 
POS completers. 

The preceding may explain the outcomes between POS completers and everyone else, but CTE 
concentrators also outperfonned POS completers on some measures. One explanation for this 
came from district research office personnel, who had experienced difficulties trying to identify 
appropriate comparison groups in their own research on the district’s career academies because 
so many South high school students were in POS. In our sample, CTE concentrators were 
students who were in IB, STEM, or performing arts programs who also completed a CTE 
concentration. We saw this with course data (see Identifying POS Completers, Appendix A). 

In sum, findings from the three districts indicate that taking more CTE credits may boost GPA, 
the probability of graduation, and some achievement measures. Further, based on the posthoc 
findings, these results come at little to no cost to overall academic achievement. 

Offer the Opportunity, Where Appropriate, for 
Secondary Students to Acquire Postsecondary Credits 

All three of the participating districts had credit transfer agreements in place with local 
postsecondary institutions; some were developed by the state, and others were developed locally 
by secondary and postsecondary faculty and administrators. West District and South District 
offered opportunities to earn general education and humanities credits through their dual 
enrollment programs. By contrast, East District offered STEM and CTE courses through its dual 
enrollment program. All three districts provided opportunities to earn postsecondary CTE credits 
through articulation agreements between high school and community college programs. Under 
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these agreements, high school courses could take the place of some introductory college courses. 
East District and ECC were more advanced in the use of online courses as a course delivery 
mechanism than the other districts in the study. 

In addition to these structured programs, community colleges also allowed high school students 
to enroll independently in their programs upon meeting age and placement requirements. In West 
District and East District, students could apply to their high schools to get their community 
college credits approved for high school credit. All three districts offered PLTW programs that 
included supplementary end-of-term assessments that could earn students college credit at 
PLTW-affiliated colleges and universities. 

Many students we spoke with were aware of the credit transfer opportunities at their schools, but 
not all of them were availing themselves of the opportunity to accrue college credits. We 
interviewed students in both West District and South District who told us that they had decided 
not to risk their GPAs or their college prospects on such demanding opportunities, which also 
included AP courses. Budget cuts to East District’s dual credit programs may also have affected 
participation rates there. East students who were motivated to earn postsecondary credits in high 
school seemed to value AP credits over the articulated credits with ECC. 

In no case did these credit transfer opportunities automatically bestow postsecondary credit, 
although WCC did create student transcripts while West District students were still in high 
school. However, the only way for West students to actually obtain their transcripted credits was 
to enroll at WCC. If they attended another postsecondary institution, their WCC credits were not 
available. In East District and South District, students sometimes had to take exams or otherwise 
demonstrate competency in addition to enrolling at the community college in order to obtain 
their accrued credits. 

Postsecondary Credits Outcomes 

States, districts, and education agencies have invested much time and effort encouraging high 
school students to earn college credits while still in high school. The results of these efforts were 
obvious in the students we met who planned to graduate from high school just a handful of 
college credits shy of an associate degree. Students like these were not the norm, however. As 
shown in the descriptive outcomes statistics tables for each district (cf. Tables 3.3, 4.3, and 5.3), 
the percentage of students who took advantage of these opportunities and actually accrued 
college credits was quite low, averaging about 5% across all groups and districts. 

Despite the promotion of these credit-based transfer mechanisms within POS, completing a POS 
was not associated with accruing more college credits. In fact, when there was a significant 
difference between POS completers and the other groups, as there was in West and East 
Districts, the difference favored the rest of the sample, not POS completers. The models we ran 
on college credits accrued were not as well-aligned with the data as the other analyses, so this 
may have affected the results. We kn ow from the qualitative data that all students, not just POS 
students, were encouraged to take courses that offered the opportunity to earn college credits. It 
could be that non-POS students had more time in their schedules. Or, as the qualitative data have 
shown, POS students in these districts saw themselves as bound for four-year colleges and 
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universities, and the community college credits that were being offered did not seem as useful or 
desirable to them as other college -preparatory approaches like AP and other advanced course- 
taking. 

Policymakers should note that many of the dual credits available to high school students come 
from local community colleges. In our districts, a great deal of energy was expended during the 
POS design and implementation process in bringing secondary and postsecondary faculty 
together to align their programs. But if POS are designed to align with and lead to postsecondary 
programs, and if they are designed to include standards-based coursework associated with a 
college-preparatory high school program, then it should come as no surprise that POS students 
will be less interested in community college credits that will not transfer to four- year universities 
and will be more interested in earning AP credits. Indeed, descriptive statistics in each district 
show that the percentage of students earning AP credits is much higher than the percentage of 
students accruing college credits through credit-based transfer programs like dual enrollment (cf. 
Tables 3.3, 4.3, and 5.3). 

West District. Dual enrollment was a small program in West District: Few students participated 
because not many high schools had teachers who were college-certified to teach the classes. 
College credits accrued through community college articulation agreements attracted more 
students, but POS students appeared to be more interested in four-year colleges and universities, 
often out of state, which made those credits non-transferable. 

East District. One comparison school principal asserted that dual enrollment was preferable to 
earning AP credits because dual enrollment allowed students to receive credit on a real college 
transcript. However, sample students appeared to value AP credits more (cf. Table 4.3). East 
District students were also limited in the types of courses they could take — limitations that were 
imposed by the state for budget reasons. A strong press for four- year college attendance may also 
have led to the low numbers of college credits accrued in East District. 

South District. Although dual enrollment participation in all districts was disappointing, 
participation in South District was the lowest, with low numbers of college credits accrued in all 
posthoc groups. Compared to some groups in the other two districts, South students simply were 
not taking advantage of the opportunity to accrue college credits. This may have been because 
the state increased the placement exam requirements for dual enrollment. South District 
encouraged students to take academic college courses rather than occupational courses because 
the local community college tended to schedule occupational courses as day-long blocks that 
were difficult for high school students to take. Articulation agreements existed, but articulated 
credits were not immediately available. Outcomes from this cohort did not match the positive 
rhetoric we heard about participation in these programs. 

Lead to an Industry-Recognized Credential or Certificate at the 

Postsecondary Level, or an Associate or Baccalaureate Degree 

Adults participating in this study valued industry-recognized credentials and certificates as a 
means of aligning secondary and postsecondary curricula, affirming the quality of their CTE 
programs, and providing students with stackable credentials leading to a broad range of 
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educational and career options. In addition to signaling students’ work readiness to potential 
employers, industry-recognized credentials were perceived as offering significant financial 
benefits. Students with certificates in hand could use them to get well-paying jobs to help pay for 
college. Schools also saw these credentials as a valuable recruiting tool. Despite these benefits, it 
was not always possible to offer industry-recognized credentials in POS, particularly in those 
cases in which a POS program area lacked alignment with a national industry accrediting body. 
Other barriers to offering credentials included the high costs of implementation, teacher training, 
and student testing; curricular alignment issues between high school and postsecondary 
programs; stringent technology and equipment requirements; scheduling constraints; liability and 
eligibility issues; and the perceived difficulty of certain exams. 

With the support of WCC and business and industry partners, West District sought to make 
industry-recognized credentials available in all of its POS for which a national-level, industry- 
approved accrediting agency existed. As both district and WCC participants acknowledged, 
however, not all programs aligned with national certifications; the district also lacked a 
mechanism for tracking the acquisition of credentials. 

A number of East District high schools, including Blue, featured programs with credential or 
certificate-earning potential. However, industry-recognized credentials were not a major focus of 
district policy, and many participants cited their high costs as a barrier to their implementation in 
this economically hard-hit region. 

Industry-recognized credentials were most robustly implemented and supported in South 
District. The state required all districts to create career academies that would allow all enrolled 
students to earn college credits and industry credentials. The state backed this policy with ample 
resources, professional development, and an accountability component that made completion of 
credentials a component of schools’ grades. South further committed to credential completion by 
fully funding the cost of students’ certification exams. Unfortunately, data collection lagged 
behind policy implementation, and valid, reliable numbers on the credentials that students earned 
were not available for this study. 

Industry-Recognized Credential, Certificate, or Degree Outcomes 

Depending on how the sentence in the legislation is parsed, this component of Perkins IV 
requires POS to lead to either a credential, certificate, associate degree, or a baccalaureate 
degree. Most of these credentials are earned at the postsecondary level. As noted, however, 
industry credentials have also become common in high schools and are even appearing in middle 
school programs. Collecting data on industry-recognized credentials is difficult. School 
counselors told us that many credentialing exams are proprietary. Students wishing to take the 
exams often must register on their own, not through the district, and exam results are frequently 
sent to students’ homes, not to schools. This made collecting student achievement data difficult. 

No usable industry-recognized credential data were collected in West or East District; in South 
District, the data we obtained were known to be incomplete, but provided a rough picture of 
credential-earning by POS students. About 16% of our posthoc sample took an exam for an 
industry credential. This was despite the state’s incorporation of industry credentials into school 
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accountability as well as the district’s push to fund the cost of the exams. Of those who took an 
exam and earned an industry credential, three quarters were POS completers. 

POS are also designed to lead to associate and baccalaureate degrees. Although we lacked the 
research funding to follow our cohorts into postsecondary education and careers, future research 
on this sample would provide important infonnation on the second, postsecondary half of POS. 

In spite of this limitation, senior exit survey results suggested that similar numbers of 
intervention and comparison students planned to attend a four-year college full time in both West 
and East Districts (South did not participate in the senior exit survey). This suggests that POS 
can be offered to high school students with no harm to their college aspirations or preparation. 

Notably, senior survey data showed that significantly more POS students in both West and East 
Districts indicated that their college studies would be related to their high school programs. This 
suggests that students who enroll in a POS may continue their education in the same program 
area and reap the benefits of beginning that preparation in high school. Qualitative data showed 
this to be a purposeful goal of POS: Teachers and administrators in both districts spoke of 
putting students on a fast track to college by (a) getting the foundational and pre-requisite 
courses out of the way while still in high school, and (b) allowing students to earn industry 
credentials that would help them work in their fields while continuing their studies. 

Limitations and Future Directions 

This study’s generalizability is limited in several ways. First, data did not come from a random 
sampling of the population of U.S. secondary students. Any generalization to districts that do not 
closely resemble those studied should be tentative. This limitation also extends to our posthoc 
analyses. Second, enrollment in a POS did not function well as an instrument for number of CTE 
credits earned in East and South Districts. Thus causal inference must remain tentative regarding 
the effects of earning CTE credits in these districts. One of the challenges of instrumental 
variable modeling is that good instruments can be difficult to find. Future research should 
examine school district contexts to detennine whether enrollment in POS schools can provide 
greater access to CTE credits. 

In spite of these limitations, this study provides evidence that under certain conditions, enrolling 
in POS schools may increase student graduation rates. The results provide impetus for exploring 
methods to increase the number of CTE credits that students take. Identifying the characteristics 
of districts that use enrollment in POS schools as a means of increasing the number of CTE 
credits earned could help administrators improve graduation rates. It may be that enrollment in 
POS schools must be accompanied by context-sensitive procedures (i.e., district-sensitive 
procedures) that enable students to respond by taking more CTE credits. Alternatively, some 
districts may not be good candidates for intervention via enrollment in POS. 

Discussion 

POS are complex, multi-faceted programs. The four mandated elements of Perkins IV allowed us 
to make numerous statistical comparisons of some of these facets. For instance, our quantitative 
findings indicated that POS may increase the prevalence of graduation by increasing the number 
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of CTE credits students earn. Other facets could not be quantified, but our qualitative data 
suggested they had an impact on students: Our interviews with POS seniors across these districts 
revealed their impressive interpersonal skills, leadership abilities, and ease in describing their life 
goals. 

The posthoc proximal outcomes presented in this report are mixed. In general, POS had no 
negative effects on high school achievement for POS students in West or East District, and only 
one substantively negative outcome in South District (AP credits earned), a district in which POS 
students were compared to IB and STEM students — the best match available to us. The 
preponderance of positive proximal outcomes argues for examining the more distal outcomes 
that are part of POS, including postsecondary enrollment, persistence, and completion, as well as 
employment information like wages. The federal government has made a major investment in 
POS, but paid little attention to secondary POS students’ post-high school trajectories. 

The outcomes reported here are based on credits and grades earned in high school: advanced 
academic credits and CTE credits. Earning CTE credits in and of itself does not make a POS. 
Students received much more than grades and high school credits by participating in POS. As 
described in this report, students received career information and preparation, hands-on, 
challenging coursework developed with industry input and aligned with state academic standards 
and postsecondary programs, and leadership and internship experiences that helped them develop 
into mature, thoughtful people with plans for next steps after high school. We did not see this to 
the same extent in our interviews with comparison school students. Qualitative findings showed 
that, whether or not they earned similar numbers of credits in the various categories we 
examined, POS students were taking advantage of other, less measurable aspects of POS, like the 
sense of identity and support provided by POS, the close relationships between teachers and 
students, and the strong career preparation that put them ahead of students with a more 
traditional high school experience. 

We concluded that most students enjoyed participating in POS, and chose their POS based on 
interest, if not zeal. They also seemed to recognize that their choices were not “forever,” but 
instead offered opportunities to decide whether a particular career was right for them or not. 

“You really learn what you want to do,” said one POS student. We met many students who were 
committed to their POS theme and who were changing their lives based on that POS. POS 
allowed them to see who they could potentially become and how to get there. 

POS differ from the traditional high school experience in several ways. By grouping together 
students with like interests, POS help students feel a sense of belonging in a program or school 
dedicated to those interests. This stands in stark contrast to the often chaotic, disengaging 
atmosphere in many comprehensive high schools (Bridgeland et al., 2006; Powell et al., 1985). 
Given the POS application and selection process, students also may feel validated or at least 
fortunate to have been selected. POS allow students to pursue their interests in depth. 

As a curriculum, POS are not unique in these differences. Districts have developed other choice 
and magnet programs precisely because they provide different experiences that engage students. 
But POS do differ from other choice options in their career focus. POS help students get a head 
start on college and career, provide them with real work experiences like internships, and allow 
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young people to imagine and plan for their futures. A POS may be the only place in which some 
low-income and minority students are held to the expectation that they will go to college; this 
may help them overcome obstacles that may exist in other aspects of their lives. 

Some elements of POS were evident in the best examples of CTE programs at comprehensive 
high schools. In West District, Amaranth’s culinary and auto programs had healthy CTSO 
participation, and student leadership was fostered there. Similarly, in East District, Emerald’s 
culinary program was a regular champion in regional and state FCCLA competitions. However, 
we also observed that other elements of POS were not offered in regular CTE programs. 
Compared to POS teachers, comprehensive high school teachers had fewer relationships with 
community college or university faculty and businesspeople. 

POS completers have intensive course experiences that integrate academic and technical 
knowledge and skills within hands-on projects that focus on solving real problems both in the 
classroom and at work sites. Work-based learning experiences like job shadowing and 
internships are continually reinforced with ongoing career exploration and development activities 
like resume preparation, job searches, portfolio presentations, and mock interviews. POS 
students also receive college and career guidance from a range of caring adults including their 
counselors and their POS teachers. Many of their POS teachers come from industry backgrounds 
and thus have a more intimate knowledge of the field. 

Senior exit survey results showed that POS offer a clear pathway to college. There were no 
differences between POS and non-POS respondents regarding their desire to attend college. 
Further, more students indicated that they wanted to attend a four-year college or university than 
a two-year institution. 

Our interview data showed that many POS students were engaged in their studies and knew what 
their next steps would be after high school. Students were purposeful about their schooling 
because they had a transition plan. Interviews with comparison school students did not show this 
to the same extent. Did POS attract high-achieving, purposeful students, or did they develop 
them? The answer to both questions may be yes: Although we talked to students whose 
participation in their POS was the fulfillment of a lifelong dream, we also spoke to students who 
chose their POS only to explore a career or supplement other interests. Both kinds of students 
were leaving high school with a strong sense of what they would do next. 

POS also affected teachers and school staff. In particular, POS changed school schedules, 
involved extracurricular activities, required ongoing professional development, and offered 
opportunities to forge partnerships with local businesses and postsecondary institutions. We saw 
veteran teachers excited about new technologies for the classroom, adopting new roles, and 
growing in their own careers. 

Although achievement outcomes did not show a consistent advantage to POS participation, they 
also did not show a disadvantage to participation. Programs of study are a worthwhile high 
school choice option that offer valuable knowledge and experiences. However, the final measure 
of the value of POS will only be known after high school (cf. Kemple & Willner, 2008). What 
kinds of college majors do POS students choose compared to other students? Do they complete 
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their degree or certificate programs faster than non-POS students? What careers do they enter, 
and what are their salaries compared to non-POS students? Does their coherent trajectory toward 
a career have other beneficial effects in terms of family fonnation, mental health, or other 
measures of young adulthood? Answers to these questions will determine the value of POS, but 
they were beyond the scope of this study. 
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CHAPTER SEVEN: Recommendations 


Recommendations emanating from the results of this study are multi-faceted. The sections below 
offer recommendations at the district, school, and policy levels. 

District-Level Recommendations 

Our primary recommendation is for districts to seek ways to increase the number of CTE credits 
earned by high school students. Our findings for West District suggested that enrollment in POS 
schools caused more students to graduate by increasing the number of CTE credits they earned. 
In all three districts, earning more CTE credits was associated with graduation. In contexts 
similar to West District, increasing opportunities for students to enroll in POS could improve 
graduation rates. In districts where POS enrollment is unlikely to change CTE credit earning, 
using other means to increase the number of CTE credits earned by secondary students could 
improve graduation rates. 

We conclude that CTE credits need not be presented in a POS sequence to have the desired 
impact on graduation. However, posthoc analyses showed that in most cases, CTE offered in 
POS sequences may have had a more beneficial impact on other important student outcomes, 
more than simply taking CTE courses or completing a CTE concentration. Further, qualitative 
data showed that most students, teachers, and administrators were positively inclined toward 
POS. They believed that POS provided students with experiences and opportunities that 
encouraged engagement and prepared them for their next steps after graduation. Administrators 
and policymakers should consider designing programs around the elements of POS that make the 
most sense for their contexts, then work to improve practice, learning, and support systems (e.g., 
data collection methods) to provide evidence of student outcomes. 

In South District, the rest of the sample outperformed POS completers on some advanced 
academic course-taking measures. These findings are likely attributable to non-ignorable 
background characteristics that contributed to student self-selection into advanced courses. The 
apparent lesson here is that if a district or state wants to encourage more STEM and AP course- 
taking, then they should do so, but not necessarily through the mechanism of POS. We found 
that POS may encourage students to take more advanced academic courses, as with STEM 
credits in West District and AP credits in East District, but South District students in STEM and 
IB programs were more likely to complete more advanced academic coursework than students in 
POS. The goal of POS should be to implement POS well, which may stimulate advanced 
academic course-taking and career preparation. POS students should not be expected to take 
more advanced academics than students who choose advanced academic programs like IB. 

Study outcomes suggest that dual enrollment and articulated programs with postsecondary 
institutions need to be re-examined. Across districts, participation in these credit -based transfer 
programs was very low. Although this may have arisen from state or district actions that limited 
participation, like eligibility requirements, credit-in-escrow policies, or a lack of resources for 
tuition or fees, we observed that community college credits appeared to have little value to 
students. Students who are interested in accruing college credits while in high school appear to 
aspire to attend four-year colleges or universities. There is a hierarchy in higher education, and 
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the students in this study, to the extent that they aspired to attend college, were choosing higher- 
status four- year institutions over community colleges. States should consider developing better 
marketing campaigns to educate university-bound students about the value of dual credit in 
community college transfer pathways. For students interested in less than a baccalaureate degree, 
states should consider redesigning credit-based transfer programs around modularized, career- 
focused certification, credential, and associate degree programs. 

Qualitative study findings also suggest that states and districts should recruit more high-quality 
teacher candidates from business and industry backgrounds. This will require changes to 
teachers’ work conditions. Business and industry teachers may need to be paid a salary more 
commensurate with professionals in the fields from which they came. Alternatively certified 
business and industry teachers also need pre-service and in-service professional development and 
support in order to plan standards-based instruction, use research-based instructional strategies, 
assess students to promote learning, and effectively manage their classrooms. All teachers need 
to be provided the same respect that other professionals have in their workplaces. 

West District administrators shared lessons learned that could help other districts implement 
POS. One official who helped plan the district’s POS high schools believed that POS could be 
well implemented in comprehensive high schools as long as they included the right teacher, a 
project-based curriculum, and the industry-standard technology and equipment needed in many 
POS. This administrator believed that the teacher and creative process fostered by project-based 
learning were the most important elements of successful POS. He also recommended that 
program standards be developed at the inception of a POS: “Not just equipment standards, not 
just textbook standards, not just professional development, but everything you need to deliver 
that program: the student certifications, the cost of student assessments, the consumable supplies, 
the tools, every piece.” 

POS may be more expensive than regular high school or CTE programs because the project- 
based, hands-on learning approaches that they emphasize require more equipment, more 
professional development, and more instructional time. Without state and district support, POS 
cannot be implemented fully. State or local bonds and taxes are some of the strategies that study 
districts used to build proper facilities for and fund POS. 

Data, Data, Data 

The future of POS could be endangered by a lack of student achievement data on POS-specific 
outcomes (i.e., industry-recognized credentials earned). POS need a broader set of outcome 
measures because there is much more to POS than traditional academic achievement. Therefore 
there is a great need to better identify POS courses, their related academic courses, and other 
POS elements that are not currently flagged in student information systems, so that these 
variables are available for analysis. 

Researchers cannot be confident of their results and recommendations if they are not confident in 
the data. In our study, we used data obtained from district research offices, the data they 
themselves use. In many cases, we found gaps in these data, suggesting that neither we nor the 
district knew the full picture. In general, we attempted to preserve sample characteristics (e.g., 
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variances) by using modem approaches to missing data handling. However, if POS are to be 
evaluated, student information systems need to catch up with POS and require the collection of 
the requisite data in order to be able to tell the story of POS and better evaluate its efficacy. 

In both West and East Districts, systems data did not include fields that would allow us, or others 
interested in POS, to track specific POS outcomes. For example, neither district included a 
variable that identified students who had enrolled in or completed a POS. Nor did either district 
include a variable identifying students who earned an industry-recognized credential. South 
District included these two data fields in their system, but the reliability of these data was suspect 
and led to the underestimated outcomes described elsewhere in this report. 

Even if we could be completely confident in the validity of the data as provided, we still faced 
what we began to call the “snapshot” effect. Many of the variables we received only reflected 
students’ status at a fixed point in time, the date on which a clerk entered attendance or 
withdrawal data into district records. These conditions could have changed the very next day. In 
urban districts with high student mobility rates, “reality” is much more fluid than such snapshots 
can capture. District information systems need to be modernized so that school-level changes 
(e.g., a student dropping a program or returning after an extended absence) can be quickly and 
frequently updated at the district level. 

Future research on POS or any education intervention rests on reliable data collection from 
schools, districts, and states. Technology exists to track progress and outcomes more efficiently 
than is presently the case. Misplaced fears have limited the collection of individual student data, 
even when those data are not personally identifiable, and this can hinder efforts to evaluate 
programs and make wise policy decisions. 

School-Level Recommendations 

Schools need not create their own POS; they may also choose to adopt a rigorous, research-based 
national curriculum like PLTW or a NAF academy. However, schools should beware of the 
temptation to implement a “POS in a box,” in which the curriculum, course sequences, 
assessments, and teacher professional development are all provided. POS, like all education 
interventions, must be allowed to develop a local flavor. For POS, this means insuring that 
business and industry are involved in developing, approving, and supporting programs through 
advisory board participation, supporting internships, and facilitating CTSOs. Like any change 
effort, people become involved and gain ownership when reforms are co-constructed, meaning 
that schools can adapt and modify the intervention to fit their unique context and mission 
(Datnow, Hubbard, & Mehan, 2002). 

Wherever possible, students should be cohorted in their academic and CTE classes. Cohorting 
helps develop a shared identity and creates a familial experience that encourages bonding, the 
mentoring of younger students, and engagement in all curricular areas. Short-term economic 
decisions to eliminate electives or place non-POS students in classes intended for POS students 
compromise the goal of cohorting and the sense of shared identity that POS foster. 
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For many high schools, alumni may offer a powerful and untapped resource. Many alumni desire 
to maintain a connection with their alma mater and enjoy serving on advisory boards and 
mentoring current students. At one comparison high school, alumni began a foundation with seed 
money that now continues to grow through an annual fund drive. That foundation pays for 
teacher professional development, special projects, and student trips to CTSO competitions. We 
also observed young alumni visiting schools to share their experiences with their former 
classmates. Perhaps even more so than teachers with industry backgrounds, young alumni with 
contemporary and compelling stories about how their high school experiences prepared them for 
success in college and work may help to motivate students to aspire to achieve the same. 

POS offer new roles for both teachers and counselors, including business partner liaisons, 
academy coordinators, and internship coordinators. We observed that teachers and counselors 
did not have to have CTE backgrounds to perform these duties effectively: Acquired familiarity 
with the programs and the desire to participate appeared more important. 

Study findings suggest that counselors for POS students should be assigned by academy or 
career pathway area, not by student last name. When counselors are assigned by academy or 
pathway, they become specialists in the subject matter taught in that academy or pathway, in part 
by participating in externships to leam about related careers. Better informed counselors are 
better able to serve students: They have a better understanding of the appropriate postsecondary 
institutions for students to attend because they have a better sense of what students are working 
to achieve. The counselors we observed who were more knowledgeable about the POS in which 
their students were participating tended to be those who were assigned to academies or pathways 
rather than to a caseload of students with disparate goals whose only similarity was a last name 
starting with the same letter of the alphabet. 

We found that POS students often received more career guidance and counseling from their POS 
teachers, who often came from industry backgrounds, than from their school counselors. We note 
two limitations to locating career guidance in the classroom as a CTE teacher responsibility. 

First, doing so assumes that non-CTE students do not need career guidance. Second, the practice 
ignores the fact that not every student enrolled in a POS is planning to pursue a career in that 
area; such students may need career guidance of a different or more general type. Career 
guidance and counseling should be a school-wide, distributed responsibility that takes place in 
every classroom as well as in extracurricular, co-curricular, and work-based learning contexts. 

Middle schools, especially middle school counselors, also have a role to play. In districts with 
lotteries or school choice programs, students must choose POS or other programs while they are 
in eighth grade. Middle school students need intensive guidance counseling as well as career 
development and exploratory experiences in order to understand their choices and the processes 
involved in applying for high school programs. It is also vitally important that middle school 
students arrive prepared for more rigorous coursework. As such, middle schools need to offer 
appropriate advisement as to the courses students need to be ready for high school. 

Study outcomes show that CTE engages students and gives them a reason to stay in school. 
Analyses show that CTE, including POS participation and completion of an intensive course 
sequence, can contribute to the likelihood of graduation. Given this finding, it may be 
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counterproductive to take away CTE and “double up” low-achieving students on academic 
classes, as is current practice in some districts. Some of the schools participating in this study 
created block schedules that allowed struggling students to double up on their academic classes 
without harming their ability to take electives like CTE. 

Other Policy Recommendations 

As plans and blueprints are drawn for the future of Perkins and CTE in the United States, we 
urge policymakers and educators to remember that CTE credit-earning is a positive path to high 
school graduation. Completing a strong CTE course sequence does not hinder college aspirations 
or attendance; indeed, for some students, it awakens such goals. Further, other elements of POS 
provide a framework that supports college-going, including standards-based academics, seamless 
transitions to postsecondary programs, college and career development and exploration, and 
opportunities to earn industry-recognized credentials and participate in work-based learning. 

Seemingly unrelated state education policies can in fact affect POS: For example, a state policy 
requiring class size reductions in one category of courses (i.e., core academic courses) will 
increase class sizes in other categories of courses. In South District, a class size mandate 
prevented most schools from cohorting their POS students in academic and POS courses. State 
legislatures need to consider the ramifications of policies before enacting them. Many school- 
level participants told us that they were compelled to follow policies even when they seemed 
whimsical, politically driven, or made in ignorance of the realities of contemporary education. 

It would be difficult for any district to implement all elements of POS well. The districts in this 
study emphasized different POS elements according to their unique abilities, state accountability 
contexts, and cultures. West District focused on advisory committee input, strong articulations 
with higher education, and the comparatively easy accrual of college credits. South District 
focused on industry-recognized credentials and work-based learning experiences, namely student 
internships. More than the other two districts, East District’s efforts to ensure that students were 
being prepared to go to college were dampened by the harsh effects of the recession. Local 
contexts are a key driver in how education interventions are implemented and the outcomes that 
are achieved. We therefore recommend that any further research on POS employ a mixed- 
method design like ours as a means of interpreting those outcomes. 

Final Comment 

The Perkins IV interim assessment report asserted that it is “feasible to provide a sound 
academic and additional technical education at the secondary level” (U.S. Department of 
Education, 2013b, p. xxv). This study confirms that although high-quality CTE programs in the 
form of POS are not easy, cheap, or capable of solving all educational problems, they can be 
implemented well and yield positive results, as the three districts that participated in this study 
demonstrated. However, this study remains an incomplete examination of POS because the 
postsecondary portion was not able to be incorporated here. Future work must include the 
complete intervention in order to arrive at a comprehensive understanding of the impact of POS 
on student secondary, postsecondary, and workplace outcomes. 
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APPENDIX A: Quantitative Technical Appendix 


This appendix provides more detail and supplemental tables to support the outcomes in the main 
body of the report. 


Instrumental Variable Estimation 

The following tables display the regression results and significant predictors for the GPA and 
graduation instrumental variable analyses for each district. They provide infonnation on the 
unstandardized and standardized regression coefficients as well as the covariances and 
correlations used in the body of the report. 
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Table A1 

Unstandardized and standardized regression coefficients, West District 


Regression Coefficients 


GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

# CTE Credits 

# CTE Credits 


Age 
Gender 
F/RL eligible 
LEP 

Special Ed IEP 
Discipline occurrences 
Asian 

Native American 

Black 

Latino 

Grade 8 pre-test math scale score 
Grade 8 pre-test science scale score 
Grade 8 pre-test reading scale score 

# CTE credits earned 
Age 

Gender 
F/RL eligible 
LEP 

Special Ed IEP 
Discipline occurrences 
Asian 

Native American 

Black 

Latino 

Grade 8 pre-test math scale score 
Grade 8 pre-test science scale score 
Grade 8 pre-test reading scale score 

# CTE credits earned 
POS school 
Gender 


b 

SE 

P 

P 

-.024 

.026 

.356 

-.019 

-.145 

.029 

.000* 

-.108 

-.139 

.032 

.000* 

-.094 

-.203 

.079 

.010* 

-.048 

-.458 

.077 

.000* 

-.137 

-.091 

.012 

.000* 

-.174 

.161 

.042 

.000* 

.088 

-.180 

.262 

.492 

-.023 

-.198 

.054 

.000* 

-.088 

-.247 

.034 

.000* 

-.176 

.003 

.000 

.000* 

.213 

.003 

.000 

.000* 

.060 

.001 

.000 

.035* 

.040 

-.001 

.010 

.959 

-.002 

-.252 

.076 

* 

o 

p 

-.128 

-.047 

.084 

.574 

-.022 

-.189 

.088 

.031* 

-.081 

-.222 

.235 

.344 

-.033 

-.173 

.176 

.327 

-.033 

-.085 

.028 

.002* 

-.174 

.272 

.119 

.022* 

-.104 

.568 

.617 

.358 

.095 

.190 

.141 

.176 

.047 

.326 

.095 

.001* 

.054 

.001 

.001 

.162 

.087 

.002 

.001 

.080 

.106 

-.001 

.001 

.480 

-.052 

.121 

.028 

.000* 

.314 

2.817 

.124 

.000* 

.514 

.378 

.107 

.000* 

.069 


Indirect Effects 


GPA <- 

POS school 

.028 

.021 

.257 

.021 

Graduation <— 

POS school 

.354 

.082 

.000* 

.166 

Covariances and Correlations 

cov 

SE 

p 

r 

# CTE Credits <-*• 

GPA 

.338 

.062 

.000* 

.242 

# CTE Credits <-*• 

Graduation 

.607 

.161 

.000* 

.265 

GPA <-*■ 

Graduation 

.191 

.021 

.000* 

.313 


*Denotes statistically significant at p < .05. 
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Table A2 

Unstandardized and standardized regression coefficients, East District 


Regression Coefficients 



b 

SE 

P 

P 

Gender 

-.352 

.027 

.000* 

-.201 

Over age 

-.487 

.145 

.001* 

-.064 

F/RE eligible 

-.136 

.027 

.000* 

-.073 

LEP 

.083 

.081 

.309 

.020 

Special Ed IEP 

-.112 

.113 

.322 

-.019 

Discipline occurrences 

-.093 

.005 

.000* 

-.139 

Asian 

.255 

.135 

.058 

.055 

Native American 

.500 

.623 

.422 

.024 

Black 

.028 

.069 

.414 

.015 

Latino 

-.135 

.060 

.024* 

-.047 

Grade 8 pre-test math scale score 

.045 

.004 

.000* 

.378 

Grade 8 pre-test reading scale score 

.013 

.004 

* 

o 

p 

.106 

Grade 8 pre-test science scale score 

.009 

.004 

.018* 

.076 

# CTE credits earned 

.147 

.009 

.000* 

.337 

Gender 

-.182 

.062 

.003* 

-.082 

Over age 

-.604 

.599 

.314 

-.062 

F/RL eligible 

-.165 

.082 

.044* 

-.070 

LEP 

-.129 

.262 

.622 

-.024 

Special Ed IEP 

-.241 

.054 

.000* 

-.033 

Discipline occurrences 

-.078 

.037 

.035* 

-.092 

Asian 

.390 

.217 

.072 

.066 

Native American 

-.006 

1.425 

.997 

.000 

Black 

.265 

.116 

.022* 

.109 

Latino 

.177 

.282 

.530 

.049 

Grade 8 pre-test math scale score 

.013 

.004 

* 

o 

p 

.083 

Grade 8 pre-test reading scale score 

.004 

.004 

.382 

.026 

Grade 8 pre-test science scale score 

-.007 

.003 

.044 

-.046 

# CTE credits earned 

.450 

.013 

.000* 

.812 

POS school 

.004 

.044 

.921 

.001 

LEP 

-.866 

.192 

.000* 

-.091 

Discipline occurrences 

-.251 

.024 

.000* 

-.164 

Latino 

.992 

.105 

.000* 

.152 

Asian 

.663 

.189 

.000* 

.062 

Black 

.364 

.088 

.000* 

.083 

Grade 8 pre-test math scale score 

.058 

.008 

.000* 

.213 


GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

# CTE Credits 

# CTE Credits 

# CTE Credits 

# CTE Credits 

# CTE Credits 

# CTE Credits 

# CTE Credits 


Indirect Effects 


GPA 

<— POS school 

.001 

.006 

.921 

.000 

Graduation 

<— POS school 

.002 

.020 

.921 

.001 

Covariances and Correlations 

cov 

SE 

P 

r 

GPA 

<-> Graduation 

.195 

.014 

.000* 

.367 


*Denotes statistically significant at p < .05. 
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Table A3 

Unstandardized standardized regression coefficients, South Di 
Regression Coefficients 


GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

GPA 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

Graduation 

# CTE Credits 

# CTE Credits 

# CTE Credits 

# CTE Credits 


Gender 
F/RL eligible 
LEP 

Special Ed IEP 
Discipline occurrences 
Asian 

Native American 

Black 

Latino 

Grade 8 pre-test math scale score 
Grade 8 pre-test reading scale score 
Grade 8 pre-test science scale score 

# CTE credits earned 
Gender 

F/RL eligible 
LEP 

Special Ed IEP 
Discipline occurrences 
Asian 

Native American 

Black 

Latino 

Grade 8 pre-test math scale score 
Grade 8 pre-test reading scale score 
Grade 8 pre-test science scale score 

# CTE credits earned 
POS 

F/RL eligible 

Grade 8 pre-test reading scale score 
Black 


Indirect Effects 
GPA 

Graduation 


Covariances and Correlations 

GPA <-> Graduation 

*Denotes statistically significant at p < .05. 


b 

SE 

P 

P 

-.268 

.024 

.000* 

-.169 

-.138 

.034 

.000* 

-.079 

.443 

.060 

.000* 

.122 

.065 

.065 

.318 

.017 

-.179 

.028 

.000* 

-.139 

.096 

.076 

.204 

.027 

-.017 

.161 

.915 

-.002 

.030 

.035 

.377 

.015 

-.051 

.030 

.093 

-.026 

.009 

.001 

.000* 

.411 

.003 

.001 

.000* 

.207 

.001 

.000 

.000* 

.108 

.052 

.009 

.000* 

.105 

-.142 

.075 

.060* 

-.060 

-.329 

.083 

.000* 

-.127 

.040 

.115 

.729 

.007 

.152 

.187 

.416 

.027 

-.205 

.053 

.000* 

-.072 

.407 

.268 

.129 

.066 

.372 

.629 

.555 

.000 

.334 

.094 

.000* 

.109 

.064 

.120 

.596 

.049 

.011 

.002 

.000* 

.336 

.004 

.004 

.382 

.302 

.000 

.001 

.991 

-.001 

.345 

.022 

* 

o 

p 

.470 

.700 

.215 

.000* 

.217 

.424 

.089 

.000* 

.120 

-.007 

.001 

.000* 

-.177 

.260 

.024 

.002* 

.576 


.036 

.013 

.004* 

.023 

.242 

.076 

* 

o 

p 

.102 

cov 

SE 

p 

r 

.328 

.017 

.000* 

.367 
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Assessing the Effect of Completing a POS on High School Achievement: 

Posthoc Multiple Regression Analysis 

Decision Rules for Exclusion from the Posthoc Multiple Regression Sample 

For the posthoc analyses, we excluded cases rather than use the imputed sample because we 
needed to limit the analyses to students who were known to have been there all 4 years taking 
classes, in order to ensure that all students had the same opportunity to complete a POS. 

West District 

We used district-provided withdrawal codes and dates to detennine which students withdrew 
from the district during each school year. To be considered a withdrawal, a student must have 
been coded as withdrawn for that year and also be missing valid course data for that year. We 
were told that the withdrawal code field was not fully reliable, so we investigated further. We 
coded students as withdrawn if they attended fewer than the 180 possible days attended and were 
missing course data for spring of that year or had courses listed for that spring but all the grades 
were F. District policy states that students who are excessively absent receive failing grades, so 
we assumed that receiving all Fs indicated that the student had been excessively absent before 
withdrawing from the district. If we found that a student had a withdrawal code, suggesting that 
the student had left and possible days attended were less than a full year, but there was valid 
course data for that year and the following year, the student was not coded as withdrawn. If a 
student had a withdrawal code of withdrawn and had full data for that year, but was missing data 
in the fall of the following year, the withdrawal code was attached to the following year. 

Students were kept in the analysis only if they had been in the district all four years. There were 
cases where the district could not locate transcript data. Therefore, if a student had remained in 
the district all four years but was missing course data, that student was excluded from the 
posthoc multiple regression analyses. Lastly, we excluded students who were missing baseline 
math and reading achievement data as those variables served as predictors for our posthoc 
multiple regressions. 

East District 

We used district-provided withdrawal codes and dates to detennine which students withdrew 
from the district during each school year. To be considered a withdrawal, a student must have 
been coded as withdrawn for that year and also be missing valid course data for that year. As 
with West District, we were infonned that the withdrawal code field at East District was not fully 
reliable, causing us to investigate further. We coded students as withdrawn if they attended fewer 
than the 180 possible days attended and they were missing course data for spring of that year or 
had courses listed for that spring but all the grades were F. Like West, East District policy 
includes excessive absenteeism as grounds for course failure, so in those cases where a student 
received all Fs, we assumed that the student had been excessively absent before withdrawing 
from the district. If a student had a withdrawal code suggesting that the student had left and 
possible days attended were less than a full year, but there was valid course data for that year and 
the following year, the student was not coded as withdrawn. If a student had a withdrawal code 
of withdrawn and had full data for that year but was missing data in the fall of the following 
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year, the withdrawal code was attached to the following year. Students were kept in the analysis 
only if they had been in the district all four years. There were two exceptions to this rule: (1) a 
student who had completed the credit requirements for graduation and spent the fourth year 
abroad, and (2) two students who had earned enough credits to graduate at the end of 1 1 th grade. 
Those three students remained in the analyses. 

South District 

First, we only included students from our district data who were eligible for at least one of the 
POS applied for. We could not detennine annual withdrawal information as South district did not 
send yearly withdrawal data. Therefore, we had to use course data to determine who should be 
excluded from the posthoc analyses. Because semester infonnation was not given to us, we could 
only determine what year a course was taken, not which semester as in the other participating 
districts. If a student was missing one year of data, that student was excluded. If the data 
indicated that a student had graduated, but we had fewer than the required number of credits for 
graduation in our data, that student was excluded under the assumption that we were missing 
data. If a student did not graduate and attended all four years but our data showed fewer credits 
attempted than the graduation requirements, we excluded that student under the assumption that 
we were missing data for that student. Of those who were not missing course data, to remain in 
our analyses, students had to have complete data for 0809 school name and 0809 grade level to 
ensure that these students were our cohort of interest. Lastly, we had to exclude those who were 
missing baseline math and reading achievement data as those variables served as predictors for 
our posthoc multiple regressions. 


Identifying POS Completers 

Like many districts, neither West nor East District had fields in their systems data to identify 
students who completed a POS. West and South Districts had fields for ninth grade assignment 
to a POS, and for East District, starting ninth grade at Blue Academy was equivalent to 
assignment for ninth grade, but student mobility in and out of POS and POS high schools made 
those fields inadequate for detennining which students completed a POS or a CTE concentration. 
A field indicating participation in a career academy was provided by South District, but it was 
incomplete and ultimately not used. Instead, we looked to empirical student course-taking and 
district-provided data on required course sequences or courses to assign students to the 
categories of POS completer or CTE concentrator, respectively. 

West District 

Files were created for students who attended each of the three POS high schools as well as the 
other POS and magnet high schools where students could earn a POS (see Sampling 
Procedures). We used the district’s POS course sequence sheets as well as high school course 
catalogs to identify which courses were required for each POS. The district CTE office provided 
a list of exit courses which we used to identify which capstone courses were required for each 
POS. In the multiple records data, each course in a POS was dummy coded for that POS. We 
also examined student course-taking to identify additional courses used to complete a POS that 
were not on the course sequence sheets. We used the list of exit courses to determine the exit 
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course for each POS or CTE concentration. Then a “respective credits earned” variable was 
created for each course. The sum of credits earned per POS was aggregated to determine if the 
required number of credits per POS was met. 

Similar steps were followed to determine if a student completed a CTE concentration: The 
district required two credits in a program area to complete a CTE concentration in that area, with 
one of those credits being from an exit level course. Having identified the necessary courses for 
each POS, we examined each course record, and for those intervention students who did not 
meet the course requirements for completing a POS, we identified the POS they had attempted. 
We did so by examining their course sequences in the multiple records format. If the course- 
taking did not definitively indicate which POS was attempted, we looked to the POS to which 
the student was originally assigned. With these data, we were able to identify the following three 
groups: (1) POS Completers, defined as students who completed a POS; (2) All Others, defined 
as all those who did not complete a POS; and (3) CTE Concentrators, defined as those who did 
not complete a POS but did complete a CTE concentration. The CTE Concentrators are a 
subgroup of All Others. 

East District 

We identified the required courses for each of the POS offered at Blue Academy. For classifying 
the engineering and medical POS (see Chapter 4), we utilized the course sequence sheets for 
each POS as well as the Class of 2012 course registration planning sheet used by counselors to 
place students in appropriate capstone courses. For the IT academy (see Chapter 4), we were not 
provided with course sequence sheets; therefore, we examined the IT courses taken to see if there 
were discernible course-taking patterns. We identified two such patterns that were either 
business oriented (focus on programming) or tech oriented (focus on engineering technology). 
Some of the POS course sequences in all three academies were changed during the study, such as 
an engineering program that adopted PLTW during our cohort’s years in high school. In other 
cases, budget limitations prevented all capstone courses from being offered. Our contextual 
knowledge of these events helped us interpret the course-taking patterns we encountered in the 
data. 

There were instances in which the constellation of courses could result in more than one POS if 
one or more courses could count toward more than one POS, but we only assigned more than one 
POS if the student took enough courses such that no course counted toward both POS (“double 
dipping”). If we could not assign more than one POS, we noted when the courses were taken and 
assigned the POS to the first course taken in that sequence, according to the year or even the 
semester within the year. 

To detennine whether a student completed a CTE concentration, we started with a district- 
provided document that identified which courses counted toward which CTE program area. 
Because we had courses in the data that were not listed on this document, we spoke with the 
district’s CTE coordinator, who helped us identify for which concentration a course counted and 
whether it was a capstone course. Four credits were needed to complete a concentration, one of 
which had to be a capstone course. Having identified the necessary courses for each POS, we 
examined each course record and for those intervention students who did not meet the course 
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requirements for completing a POS, we identified the POS they had attempted. We did so by 
examining their course-taking in the multiple records fonnat. With these data, we were able to 
identify the following three groups: (1) POS Completers, defined as students who completed a 
POS; (2) All Others, defined as all those who did not complete a POS; and (3) CTE 
Concentrators, defined as those who did not complete a POS but did complete a CTE 
concentration. The CTE Concentrators are a subgroup of All Others. For the analyses in Chapter 
4, we used a smaller subgroup still: we only included CTE Concentrators from Blue Academy 
(i.e., students who did not complete a POS) in order to explore the school effect at East District. 

South District 

To assign POS completion at South, we started with the course sequence sheets provided by the 
district. We also looked at course-taking data as well as school catalogs and websites to 
detennine sequences that were not listed on the course sequence sheets. The district required that 
students earn 4 credits for most POS, except in cases where one of the required courses was only 
.5 credits (e.g., EKG Aide). For the fourth class, the district allowed an internship, OJT (on the 
job training with a classroom component), or an independent study course to serve as the 
capstone requirement if it was related to the POS. Syntax was written to identify all POS 
completers. 

CTE concentration was detennined in two ways. First, we looked at those who did not complete 
the requirements for the POS but took enough credits in that POS to complete a CTE 
concentration. To earn a CTE concentration within a POS sequence, students earned 3 credits 
except in the case of EKG aide where they earned 2.5 credits because one of the required courses 
was only 0.5 credits. We also identified students who completed a CTE concentration in a non- 
POS course sequence (i.e., visual technology, graphic design). These were CTE programs, but 
the district did not have POS in these areas. In studying the transcript data, we saw course-taking 
sequences that met the 3 -credit requirement for a CTE concentration. These courses were related 
but not in a required POS sequence. If a POS was completed, the POS attempted was the same as 
what was completed. If not, we looked at course-taking (i.e., what was taken and when) and the 
POS to which the student was assigned. We disallowed double dipping: If the same course had to 
be used to meet the requirements for completing a POS and a CTE concentration, the POS 
completion remained and the concentration was coded to 0. With these data, we were able to 
identify the following three groups: (1) POS completers defined as those who completed a POS; 
(2) All Others, defined as those who did not complete a POS; and (3) CTE Concentrators, 
defined as those who did not complete a POS but did complete a CTE concentration. The CTE 
Concentrators are a subgroup of All Others. It is important to remember that the All Others 
group includes students from IB and other advanced academic programs — clearly these students 
were completing CTE concentrations as well as their chosen programs. 

Full Regression Tables 

West District. Following are the full regression tables, upon which the outcomes displayed in 
Tables 3.4 and 3.5 of Chapter 3 are based. We present the tables for POS completers versus all 
others and then for POS completers versus CTE concentrators. 
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Table A4 


Linear regression predicting weighted overall GPA, POS completers versus all others, West 
District 


Variable 

Adjusted R 2 
b SEb 

Overall GPA 
n = 1,368 
= 0.385 

fi P 

95% CL 
lower upper 

(Constant) 

1.387 

.102 


.000 

1.187 

1.587 

POS all others (l=POS Completer) 

All 

.029 

.142 

.000 

.121 

.232 

Azure 

-.276 

.054 

-.116 

.000 

-.381 

-.171 

Sky 

-.145 

.039 

-.086 

.000 

-.222 

-.068 

Male 

-.169 

.028 

-.137 

.000 

-.223 

-.114 

Asian 

.087 

.039 

.052 

.027 

.010 

.163 

Black 

-.091 

.047 

-.045 

.053 

-.183 

.001 

Latino 

-.075 

.033 

-.059 

.025 

-.141 

-.010 

Other 

i 

o 

o 

.140 

-.015 

.474 

-.375 

.174 

F/RL eligible 

-.031 

.031 

-.023 

.327 

-.092 

.031 

Special Education 

.005 

.071 

.002 

.947 

-.134 

.143 

LEP 

.257 

.095 

.061 

.007 

.071 

.443 

Grade 8 pre-test math 

.003 

.000 

.378 

.000 

.003 

.004 

Grade 8 pre-test reading 

.003 

.000 

.225 

.000 

.002 

.003 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 


Table A5 


Linear regression predicting STEM credits earned, POS completers versus all others, West 
District 


Variable 

STEM Credits 
n = 1,368 

Adjusted R 2 = 0.268 
b SE b fi p 

95% Cl 
lower upper 

(Constant) 

.899 

.287 


.002 

.336 

1.462 

POS all others (l=POS Completer) 

.276 

.102 

.067 

.007 

.075 

.477 

Azure 

-.219 

.192 

-.028 

.255 

-.597 

.158 

Sky 

-1.006 

.141 

-.181 

.000 

-1.283 

-.730 

Male 

-.037 

.097 

-.009 

.702 

-.227 

.153 

Asian 

.601 

.140 

.110 

.000 

.326 

.876 

Black 

-.033 

.168 

-.005 

.843 

-.364 

.297 

Latino 

.103 

.120 

.024 

.390 

-.132 

.339 

Other 

-.785 

.503 

-.036 

.119 

-1.772 

.202 

F/RL eligible 

-.125 

.112 

-.028 

.266 

-.345 

.095 

Special Education 

.224 

.253 

.022 

.375 

-.272 

.720 

LEP 

.613 

.340 

.044 

.071 

-.053 

1.279 

Grade 8 pre-test math 

.012 

.001 

.439 

.000 

.010 

.013 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 
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Table A6 

Linear regression predicting AP credits earned, POS completers versus all others, West District 

AP Credits 
n = 1,368 

Adjusted R 2 = 0.257 95% Cl 


Variable 

b 

SEb 

fi 

P 

lower 

upper 

(Constant) 

-4.745 

.436 


.000 

-5.600 

-3.891 

POS all others (l=POS Completer) 

-.146 

.122 

-.030 

.232 

-.385 

.093 

Azure 

.752 

.229 

.081 

.001 

.303 

1.201 

Sky 

-.516 

.168 

-.079 

.002 

-.846 

-.187 

Male 

.026 

.118 

.005 

.828 

-.206 

.258 

Asian 

.967 

.167 

.150 

.000 

.640 

1.295 

Black 

.089 

.200 

.011 

.657 

-.304 

.482 

Latino 

.256 

.143 

.051 

.074 

-.025 

.536 

Other 

-.277 

.599 

-.011 

.644 

-1.452 

.898 

F/RL eligible 

-.155 

.134 

-.029 

.245 

-.417 

.107 

Special Education 

.548 

.302 

.046 

.070 

-.045 

1.142 

LEP 

.596 

.405 

.036 

.142 

-.199 

1.391 

Grade 8 pre-test math 

.009 

.001 

.294 

.000 

.007 

.011 

Grade 8 pre-test reading 

.008 

.001 

.188 

.000 

.005 

.011 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 


Table A7 


Linear regression predicting college credits accrued, POS completers versus all others, West 
District 


Variable 

b 

College Credits Accrued 
n = 1,368 

Adjusted R 2 = 0.021 

SE b fi p 

95% Cl 
lower upper 

(Constant) 

2.493 

1.169 


.033 

.199 

4.787 

POS all others (l=POS Completer) 

-1.667 

.327 

-.147 

.000 

-2.308 

-1.025 

Azure 

.836 

.614 

.039 

.174 

-.369 

2.041 

Sky 

.411 

.451 

.027 

.362 

-.473 

1.295 

Male 

-.837 

.318 

-.074 

.009 

-1.460 

-.214 

Asian 

.283 

.448 

.019 

.528 

-.596 

1.161 

Black 

.118 

.538 

.006 

.827 

-.938 

1.173 

Latino 

-.541 

.384 

-.046 

.159 

-1.293 

.212 

Other 

-.925 

1.607 

-.016 

.565 

-4.077 

2.227 

F/RL eligible 

-.056 

.358 

-.005 

.876 

-.759 

.647 

Special Education 

-.919 

.811 

-.033 

.258 

-2.510 

.673 

LEP 

-.791 

1.088 

-.021 

.467 

-2.925 

1.343 

Grade 8 pre-test math 

-.001 

.003 

-.013 

.754 

-.007 

.005 

Grade 8 pre-test reading 

.000 

.004 

-.001 

.972 

-.008 

.008 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 
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Table A8 


Linear regression predicting CTE GPA, POS completers versus all others, West District 


Variable 

b 

Adjusted R 2 
SEb 

CTE GPA 
n = 1,215 
= 0.085 
fi P 

95% Cl 
lower upper 

(Constant) 

3.429 

.036 


.000 

3.358 

3.500 

POS all others (l=POS Completer) 

.155 

.038 

.121 

.000 

.082 

.229 

Azure 

.013 

.068 

.006 

.848 

-.121 

.147 

Sky 

-.036 

.050 

-.021 

.478 

-.134 

.063 

Male 

-.095 

.036 

-.074 

.009 

-.166 

-.024 

Asian 

.015 

.054 

.009 

.776 

-.090 

.121 

Black 

-.229 

.063 

-.108 

.000 

-.352 

-.106 

Latino 

-.207 

.045 

-.154 

.000 

-.296 

-.118 

Other 

-.430 

.187 

-.064 

.021 

-.797 

-.064 

F/RL eligible 

-.041 

.042 

-.029 

.332 

-.124 

.042 

Special Education 

-.527 

.091 

-.166 

.000 

-.705 

-.349 

LEP 

-.089 

.124 

-.021 

.472 

-.332 

.154 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 


Table A9 


Linear regression predicting weighted overall GPA, POS completers versus CTE concentrators, 
West District 


Variable 

Adjusted R 
b SEb 

Overall GPA 
n = 675 
= 0.391 
fi P 

95% Cl 
lower upper 

(Constant) 

1.516 

.133 


.000 

1.255 

1.777 

POS CTE Conc (l=POS Completer) 

.152 

.042 

.112 

.000 

.071 

.234 

Azure 

-.245 

.052 

-.153 

.000 

-.346 

-.143 

Sky 

-.104 

.042 

-.084 

.013 

-.185 

-.022 

Male 

-.182 

.034 

-.169 

.000 

-.248 

-.115 

Asian 

.048 

.051 

.032 

.344 

-.052 

.149 

Black 

-.135 

.057 

-.077 

.018 

-.246 

-.023 

Latino 

-.042 

.043 

-.036 

.337 

-.126 

.043 

Other 

-.102 

.152 

-.020 

.503 

-.400 

.197 

F/RL eligible 

-.015 

.040 

-.012 

.710 

-.094 

.064 

Special Education 

.083 

.099 

mi 

.402 

-.111 

.276 

LEP 

.448 

.156 

.090 

.004 

.142 

.754 

Grade 8 pre-test math 

.003 

.000 

.389 

.000 

.002 

.003 

Grade 8 pre-test reading 

.002 

.000 

.241 

.000 

.002 

.003 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 
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Table A10 


Linear regression predicting STEM credits earned, POS completers versus CTE concentrators, 
West District 


Variable 

Adjusted R 
b SEb 

STEM Credits 
n = 675 
? = 0.313 
fi P 

95% Cl 
lower upper 

(Constant) 

1.461 

.356 


.000 

.762 

2.161 

POSCTEConc (l=POS Completer) 

.541 

.143 

.123 

.000 

.261 

.821 

Azure 

.042 

.178 

.008 

.813 

-.307 

.391 

Sky 

-1.027 

.143 

-.257 

.000 

-1.307 

-.747 

Male 

-.178 

.113 

-.051 

.116 

-.401 

.044 

Asian 

.633 

.175 

.131 

.000 

.289 

.978 

Black 

-.069 

.195 

-.012 

.726 

-.452 

.315 

Latino 

.297 

.148 

.080 

.046 

.006 

.588 

Other 

-1.032 

.521 

-.064 

.048 

-2.056 

-.008 

F/RL eligible 

-.053 

.138 

-.014 

.702 

-.323 

.217 

Special Education 

.664 

.337 

.068 

.049 

.003 

1.325 

LEP 

.975 

.534 

.061 

.068 

-.073 

2.024 

Grade 8 pre-test math 

.009 

.001 

.404 

.000 

.007 

.011 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 


Table All 

Linear regression predicting AP credits earned, POS completers versus CTE concentrators, 

West District 

AP Credits 
n = 675 

Adjusted R 2 = 0.332 95% CL 


Variable 

b 

SEb 

fi 

P 

lower 

upper 

(Constant) 

-4.912 

.536 


.000 

-5.964 

-3.860 

POS CTE Conc (l=POS Completer) 

.178 

.168 

.034 

.290 

-.152 

.508 

Azure 

1.080 

.209 

.175 

.000 

.670 

1.490 

Sky 

-.399 

.168 

-.084 

.018 

-.728 

-.069 

Male 

-.191 

.136 

-.046 

.162 

-.458 

.077 

Asian 

.859 

.206 

.149 

.000 

.454 

1.263 

Black 

.324 

.230 

.048 

.159 

-.127 

.775 

Latino 

.394 

.174 

.089 

.024 

.052 

.736 

Other 

-.586 

.613 

-.031 

.339 

-1.789 

.617 

F/RL eligible 

-.121 

.162 

-.026 

.454 

-.439 

.197 

Special Education 

1.011 

.398 

.087 

.011 

.230 

1.791 

LEP 

.846 

.629 

.044 

.179 

-.389 

2.080 

Grade 8 pre-test math 

.007 

.001 

.244 

.000 

.004 

.009 

Grade 8 pre-test reading 

.010 

.002 

.260 

.000 

.007 

.014 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 
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Table A 12 


Linear regression predicting college credits accrued, POS completers versus CTE concentrators, 
West District 


Variable 

b 

College Credits Accrued 
n = 675 

Adjusted R 2 = -0.01 1 

SE b fi p 

95% CL 

lower upper 

(Constant) 

POS CTE Cone 

.572 

.335 

-.006 

.088 

-.086 

1.230 

(l=POS Completer) 

-.016 

.105 

.875 

-.223 

.190 

Azure 

.123 

.131 

.039 

.345 

-.133 

.380 

Sky 

-.063 

.105 

-.026 

.548 

-.269 

.143 

Male 

-.091 

.085 

-.044 

.283 

-.259 

.076 

Asian 

.058 

.129 

.020 

.655 

-.195 

.311 

Black 

-.094 

.144 

-.028 

.514 

-.376 

.188 

Latino 

-.056 

.109 

-.025 

.608 

-.270 

.158 

Other 

-.093 

.383 

-.010 

.809 

-.845 

.660 

F/RL eligible 

-.022 

.101 

-.010 

.825 

-.221 

.176 

Special Education 

-.132 

.249 

-.022 

.596 

-.620 

.356 

LEP 

-.090 

.393 

-.009 

.820 

-.862 

.683 

Grade 8 pre-test math 

8.220E-006 

.001 

.001 

.992 

-.002 

.002 

Grade 8 pre-test 
reading 

-.001 

.001 

-.060 

.290 

-.003 

.001 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 


Table A13 

Linear regression predicting CTE GPA, POS completers versus CTE concentrators, West 

District 

CTE GPA 
n = 675 

Adjusted R 2 = 0.066 95% CL 


Variable 

b 

SEb 

fi 

P 

lower 

upper 

(Constant) 

3.441 

.050 


.000 

3.343 

3.540 

POS CTE Conc (l=POS Completer) 

.078 

.045 

.065 

.085 

-.011 

.168 

Azure 

.048 

.056 

.034 

.394 

-.062 

.157 

Sky 

-.012 

.044 

-.011 

.796 

-.099 

.076 

Male 

-.042 

.036 

-.044 

.239 

-.112 

.028 

Asian 

.046 

.055 

.035 

.404 

-.062 

.154 

Black 

-.209 

.062 

-.136 

.001 

-.330 

-.088 

Latino 

-.182 

.047 

o 

OO 

.000 

-.274 

-.089 

Other 

OO 

O 

.166 

-.025 

.516 

-.433 

.218 

F/RL eligible 

-.039 

.044 

-.037 

.377 

-.125 

.047 

Special Education 

-.310 

.101 

-.116 

.002 

-.509 

-.111 

LEP 

.107 

.166 

.025 

.518 

-.218 

.433 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 
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East District Following are the full regression tables, upon which the outcomes displayed in 
Tables 4.4 and 4.5 of Chapter 4 are based. We present the tables for POS completers versus all 
others and then for POS completers versus Blue CTE concentrators. 

Table A14 


Linear regression predicting weighted overall GPA, POS completers versus all others, East 
District 


Variable 

Adjusted R 2 
b SEb 

Overall GPA 
n = 907 
= 0.523 
fi P 

95% Cl 

lower upper 

(Constant) 

-23.362 

1.096 


.000 

-25.513 

-21.210 

POS all others (l=POS Completer) 

.054 

.042 

.030 

.196 

-.028 

.137 

Male 

-.345 

.034 

-.234 

.000 

-.413 

-.278 

Asian 

.237 

.104 

.062 

.023 

.033 

.440 

Black 

-.045 

.062 

-.027 

.476 

-.167 

.078 

Latino 

-.065 

.081 

-.028 

.423 

-.223 

.094 

Other 

-.012 

.107 

-.003 

.909 

-.222 

.198 

F/RL eligible 

-.132 

.039 

-.085 

.001 

-.208 

-.057 

Special Education 

-.180 

.127 

-.033 

.157 

-.428 

.069 

LEP 

.166 

.097 

.045 

.089 

-.025 

.357 

Grade 8 pre-test math 

.053 

.003 

.518 

.000 

.047 

.060 

Grade 8 pre-test reading 

.019 

.003 

.187 

.000 

.013 

.026 

Note. All coefficients rounded to two digits. 

LEP = Limited English Proficiency. Cl 

= confidence 

interval. 







Table A1 5 







Linear regression predicting STEM credits earned, POS completers 

versus all others, East 

District 










STEM Credits 






n = 

907 




Adjusted R 1 

= 0.342 


95% Cl 

Variable 

b 

SEb 

fi 

P 

lower 

upper 

(Constant) 

-35.838 

2.186 


.000 

-40.128 

-31.548 

POS all others (l=POS Completer) 

.141 

.096 

.040 

.140 

-.046 

.329 

Male 

-.390 

.078 

-.137 

.000 

-.543 

-.237 

Asian 

.636 

.233 

.086 

.006 

.179 

1.093 

Black 

.265 

.138 

.084 

.055 

-.005 

.536 

Latino 

.000 

.182 

.000 

.999 

-.358 

.358 

Other 

.270 

.243 

.034 

.265 

-.206 

.747 

F/RL eligible 

-.163 

.088 

-.054 

.063 

-.335 

.009 

Special Education 

-.504 

.288 

-.048 

.080 

-1.069 

.060 

LEP 

.304 

.217 

.043 

.163 

-.123 

.731 

Grade 8 pre-test math 

.113 

.006 

.568 

.000 

.101 

.124 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 
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Table A16 


Linear regression predicting AP credits earned, POS completers versus all others, East District 


Variable 

Adjusted R 1 
b SEb 

AP Credits 
n = 907 
= 0.348 
fi P 

95% Cl 

lower upper 

(Constant) 

-50.770 

3.535 


.000 

-57.707 

-43.833 

POS all others (l=POS Completer) 

1.067 

.136 

.212 

.000 

.801 

1.333 

Male 

-.487 

.111 

-.120 

.000 

-.705 

-.269 

Asian 

.720 

.334 

.068 

.032 

.064 

1.375 

Black 

-.552 

.201 

-.123 

.006 

-.947 

-.158 

Latino 

-.413 

.260 

-.064 

.113 

-.925 

.098 

Other 

-.347 

.345 

-.031 

.315 

-1.023 

.330 

F/RL eligible 

-.339 

.125 

-.079 

.007 

-.583 

-.094 

Special Education 

.184 

.409 

.012 

.652 

-.618 

.986 

LEP 

.334 

.314 

.033 

.287 

-.281 

.949 

Grade 8 pre-test math 

.078 

.010 

.276 

.000 

.058 

.099 

Grade 8 pre-test reading 

.067 

.011 

.234 

.000 

.046 

.088 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 


Table A17 

Linear regression predicting college credits accrued, POS completers versus all others, East 

District 

College Credits Accrued 
n = 907 

Adjusted R 2 = 0.023 95% Cl 


Variable 

b 

SEb 

fi 

P 

lower 

upper 

(Constant) 

-22.493 

9.620 


.020 

-41.374 

-3.612 

POS all others (l=POS Completer) 

-.846 

.369 

-.076 

.022 

-1.571 

-.121 

Male 

-.523 

.303 

-.058 

.084 

-1.117 

.070 

Asian 

-1.325 

.909 

-.056 

.145 

-3.109 

.460 

Black 

-1.025 

.548 

-.103 

.061 

-2.100 

.049 

Latino 

-.208 

.709 

-.014 

.770 

-1.599 

1.184 

Other 

-1.759 

.938 

-.071 

.061 

-3.600 

.083 

F/RL eligible 

-.219 

.339 

-.023 

.518 

-.885 

.446 

Special Education 

-.300 

1.112 

-.009 

.788 

-2.482 

1.883 

LEP 

-1.020 

.853 

-.045 

.232 

-2.695 

.654 

Grade 8 pre-test math 

.034 

.028 

.055 

.225 

-.021 

.090 

Grade 8 pre-test reading 

.034 

.029 

.054 

.245 

-.023 

.092 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 
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Table A1 8 


Linear regression predicting CTE GPA, POS completers versus all others, East District 


Variable 

b 

Adjusted R 2 
SEb 

CTE GPA 
n = 859 
= 0.139 
fi P 

95% Cl 
lower upper 

(Constant) 

3.597 

.093 


.000 

3.413 

3.780 

POS all others (1= POS Completer) 

.027 

.064 

.014 

.667 

-.098 

.153 

Male 

-.383 

.053 

-.231 

.000 

-.487 

-.279 

Asian 

.092 

.161 

.021 

.570 

-.225 

.408 

Black 

-.550 

.094 

-.298 

.000 

-.734 

-.366 

Latino 

-.305 

.126 

-.116 

.016 

-.553 

-.057 

Other 

-.376 

All 

-.079 

.029 

-.111 

-.039 

F/RL eligible 

-.307 

.059 

-.175 

.000 

-.422 

-.192 

Special Education 

-.386 

.196 

-.063 

.049 

-.770 

-.001 

LEP 

-.125 

.146 

-.031 

.389 

-.411 

.160 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 


Table A19 

Linear regression predicting weighted overall GPA, POS completers versus Blue CTE 

concentrators, East District 

Overall GPA 
n = 226 

Adjusted R 2 = 0.463 95% Cl 


Variable 

b 

SEb 

fi 

P 

lower 

upper 

(Constant) 

-20.634 

1.931 


.000 

-24.440 

-16.828 

POSBlueCTEConc 
(l=POS Completer) 

-.012 

.088 

-.007 

.887 

so 

OO 

.161 

Male 

-.111 

.069 

-.178 

.001 

-.367 

-.096 

Asian 

.428 

.196 

.129 

.030 

.043 

.814 

Black 

-.007 

.120 

-.005 

.955 

-.243 

.230 

Latino 

-.064 

.161 

-.030 

.690 

-.381 

.253 

Other 

.373 

.225 

.092 

.099 

-.070 

.817 

F/RL eligible 

-.062 

.071 

-.045 

.387 

-.202 

.079 

Special Education 

-.683 

.293 

-.120 

.021 

-1.260 

-.105 

LEP 

.361 

.224 

.089 

.109 

-.081 

.803 

Grade 8 pre-test math 

.047 

.006 

.499 

.000 

.034 

.060 

Grade 8 pre-test 
reading 

.018 

.006 

.192 

.005 

.005 

.030 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 
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Table A20 


Linear regression predicting STEM credits earned, POS completers versus Blue CTE 
concentrators, East District 


Variable 

b 

Adjusted R 2 
SEb 

STEM Credits 
n = 226 
= 0.372 

fi P 

95% Cl 

lower upper 

(Constant) 

POS Blue CTE Cone 

-34.327 

3.742 

.032 

.000 

-41.702 

-26.952 

(l=POS Completer) 

.107 

.183 

.559 

-.254 

.469 

Male 

-.420 

.143 

-.167 

.004 

-.703 

-.138 

Asian 

.393 

.410 

.061 

.340 

-.416 

1.202 

Black 

-.215 

.250 

-.074 

.391 

-.707 

.278 

Latino 

-.449 

.337 

OC 

O 

.184 

-1.114 

.216 

Other 

OO 

O 

All 

-.014 

.819 

-1.039 

.822 

F/RL eligible 

-.201 

.150 

-.075 

.182 

-.496 

.094 

Special Education 

-.090 

.614 

-.008 

.884 

-1.299 

1.120 

LEP 

.054 

.468 

.007 

.907 

-.868 

.976 

Grade 8 pre-test math 

.110 

.010 

.601 

.000 

.090 

.130 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 


Table A21 


Linear regression predicting AP credits earned, POS completers versus Blue CTE concentrators, 
East District 


Variable 

b 

Adjusted R 
SEb 

AP Credits 
n = 226 
= 0.401 

fi P 

95% Cl 

lower upper 

(Constant) 

POS Blue CTE Cone 

-83.267 

8.038 

-.016 

.000 

-99.110 

-67.424 

(l=POS Completer) 

i 

c 

vc 

.366 

.766 

-.830 

.612 

Male 

-.941 

.286 

-.184 

.001 

-1.505 

-.376 

Asian 

2.287 

.814 

.175 

.005 

.682 

3.891 

Black 

.091 

.500 

.016 

.855 

-.894 

1.076 

Latino 

.594 

.669 

.070 

.376 

-.725 

1.914 

Other 

-.035 

.937 

-.002 

.971 

-1.881 

1.812 

F/RL eligible 

-.456 

.297 

-.084 

.126 

-1.041 

.129 

Special Education 

-.780 

1.220 

-.035 

.523 

-3.185 

1.625 

LEP 

.407 

.934 

.026 

.664 

-1.434 

2.247 

Grade 8 pre-test math 

.141 

.027 

.381 

.000 

.089 

.194 

Grade 8 pre-test 
reading 

.096 

.026 

.263 

.000 

.044 

.147 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = confidence 
interval. 
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Table A22 


Linear regression predicting college credits accrued, POS completers versus Blue CTE 
concentrators, East District 




College Credits Accrued 






n = 

226 





Adjusted R 1 

= 0.040 


95% Cl 


Variable 

b 

SEb 

fi 

p lower upper 

(Constant) 

-1.278 

3.846 


.740 

-8.858 

6.303 

POSBlueCTEConc 
(l=POS Completer) 

.225 

.175 

.088 

.201 

-.120 

.569 

Male 

.272 

.137 

.140 

.049 

.001 

.542 

Asian 

.054 

.389 

.011 

.891 

-.714 

.821 

Black 

-.632 

.239 

-.285 

.009 

-1.104 

-.161 

Latino 

-.355 

.320 

-.111 

.269 

-.986 

.276 

Other 

-.761 

.448 

-.126 

.091 

-1.644 

.123 

F/RL eligible 

-.005 

.142 

-.003 

.970 

-.285 

.275 

Special Education 

-.360 

.584 

-.043 

.538 

-1.511 

.791 

LEP 

-.353 

.447 

-.059 

.430 

-1.234 

.527 

Grade 8 pre-test math 

-.001 

.013 

-.009 

.920 

-.026 

.024 

Grade 8 pre-test reading 

.006 

.012 

.043 

.634 

-.019 

.030 

Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl 

= confidence 


interval. 







Table A23 







Linear regression predicting CTE GPA, POS completers versus Blue CTE concentrators, East 

District 










CTE GPA 






n = 

= 226 





Adjusted R 2 

= 0.096 


95% Cl 


Variable 

b 

SEb 

fi 

p lower upper 

(Constant) 

3.007 

.192 


.000 

2.628 

3.387 

POSBlueCTEConc 
(l=POS Completer) 

.202 

.119 

.111 

.090 

-.032 

.437 

Male 

.028 

.092 

.020 

.762 

-.154 

.210 

Asian 

.362 

.268 

.103 

.178 

-.166 

.891 

Black 

-.408 

.161 

-.259 

.012 

-.726 

-.089 

Latino 

-.028 

.220 

-.013 

.898 

-.463 

.406 

Other 

-.228 

.307 

-.054 

.457 

-.833 

.376 

F/RL eligible 

-.187 

.098 

-.129 

.056 

-.380 

.005 

Special Education 

-.700 

.398 

-.117 

.080 

-1.485 

.085 

LEP 

.126 

.306 

.029 

.681 

-.476 

.728 

Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl 

= confidence 



interval. 
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South District. Following are the full regression tables, upon which the outcomes displayed in 
Tables 5.4 and 5.5 of Chapter 5 are based. We present the tables for POS completers versus all 
others and then for POS completers versus CTE concentrators. 

Table A24 

Linear regression predicting weighted overall GPA, POS completers versus all others, South 

District 

Overall GPA 


Variable 

n = 2,488 

Adjusted R 2 = 0.464 
b SE b fi p 

95% Cl 

lower upper 

(Constant) 

-1.028 

.121 


.000 

-1.265 

-.792 

POS all others (1= POS Completer) 

.022 

.022 

.015 

.306 

-.020 

.064 

Male 

-.221 

.021 

-.158 

.000 

-.262 

O 

OC 

Asian 

.125 

.045 

.043 

.005 

.038 

.212 

Black 

-.011 

.032 

-.006 

.721 

-.074 

.051 

Latino 

-.065 

.028 

-.039 

.022 

-.120 

-.009 

Other 

.025 

.051 

.007 

.622 

-.075 

.126 

F/RL eligible 

-.098 

.026 

-.064 

.000 

-.149 

-.047 

Special Education 

.053 

.051 

.016 

.293 

-.046 

.153 

LEP 

.415 

.048 

.138 

.000 

.321 

.509 

Grade 8 pre-test math 

.009 

.000 

.473 

.000 

.008 

.010 

Grade 8 pre-test reading 

.004 

.000 

.241 

.000 

.003 

.005 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = Confidence 
Interval. 


Table A25 


Linear regression predicting STEM credits earned, POS completers versus all others, South 
District 


Variable 

b 

Adjusted R 1 
SEb 

STEM Credits 
n = 2,488 
= 0.372 
fi P 

95% Cl 

lower upper 

(Constant) 

-3.489 

.331 


.000 

-4.139 

-2.840 

POS all others (1= POS Completer) 

-.329 

.061 

-.088 

.000 

-.448 

-.210 

Male 

-.074 

.059 

-.020 

.212 

-.189 

.042 

Asian 

.785 

.126 

.104 

.000 

.538 

1.032 

Black 

-.007 

.090 

-.001 

.942 

-.183 

.170 

Latino 

-.113 

.080 

-.026 

.157 

-.269 

.043 

Other 

.122 

.145 

.014 

.399 

-.162 

.405 

F/RL eligible 

-.322 

.073 

-.081 

.000 

-.466 

-.179 

Special Education 

-.537 

.143 

-.062 

.000 

-.816 

-.257 

LEP 

.130 

.133 

.017 

.329 

-.131 

.392 

Grade 8 pre-test math 

.026 

.001 

.526 

.000 

.024 

.027 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = Confidence 
Interval. 
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Table A26 


Linear regression predicting AP credits earned, POS completers versus all others, South District 


Variable 

Adjusted R ? 
b SEb 

AP Credits 
n = 2,488 
= 0.457 
fi P 

95% Cl 

lower upper 

(Constant) 

-.796 

.098 

-.123 

.000 

-16.221 

-14.075 

POS all others (1= POS Completer) 

-.220 

.095 

-.035 

.000 

-.988 

-.605 

Male 

1.508 

.202 

.116 

.021 

-.407 

-.034 

Asian 

.261 

.144 

.033 

.000 

1.113 

1.904 

Black 

.175 

.128 

.023 

.070 

-.021 

.543 

Latino 

.247 

.232 

.016 

.172 

-.076 

.425 

Other 

-.231 

.117 

-.034 

.287 

-.208 

.701 

F/RL eligible 

-.126 

.230 

-.008 

.049 

-.461 

-.001 

Special Education 

1.087 

.218 

.080 

.583 

-.577 

.325 

LEP 

.030 

.002 

.352 

.000 

.660 

1.515 

Grade 8 pre-test math 

.024 

.002 

.332 

.000 

.026 

.033 

Grade 8 pre-test reading 

-.796 

.098 

-.123 

.000 

.021 

.027 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = Confidence 
Interval. 

Table A27 

Linear regression predicting college credits accrued, POS completers versus all others, South 

District 

College Credits Accrued 
n = 2,488 

Adjusted R 2 = 0.040 95% Cl 


Variable 

b 

SEb 

fi 

P 

lower 

upper 

(Constant) 

-.644 

.079 


.000 

-.799 

-.488 

POS all others (1= POS Completer) 

-.015 

.014 

-.021 

.292 

-.043 

.013 

Male 

.000 

.014 

.001 

.980 

-.027 

.027 

Asian 

.069 

.029 

.048 

.019 

.011 

.126 

Black 

.072 

.021 

.084 

.001 

.031 

.113 

Latino 

.015 

.019 

.019 

.413 

-.021 

.052 

Other 

.022 

.034 

.013 

.516 

-.044 

.088 

F/RL eligible 

-.025 

.017 

-.034 

.138 

-.059 

.008 

Special Education 

.028 

.033 

.017 

.397 

-.037 

.094 

LEP 

.069 

.032 

.047 

.029 

.007 

.131 

Grade 8 pre-test math 

.002 

.000 

.173 

.000 

.001 

.002 

Grade 8 pre-test reading 

.000 

.000 

.049 

.110 

.000 

.001 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = Confidence 
Interval. 
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Table A28 

Linear regression predicting CTE GPA, POS completers versus all others, South District 


CTE GPA 
n = 2,085 


Variable 

b 

Adjusted R 1 
SEb 

= 0.092 

fi 

P 

95% Cl 
lower upper 

(Constant) 

3.551 

.028 


.000 

3.497 

3.606 

POS all others (1= POS Completer) 

.131 

.027 

.101 

.000 

.078 

.185 

Male 

-.063 

.027 

-.049 

.021 

-.117 

-.010 

Asian 

.048 

.064 

.016 

.449 

-.077 

.174 

Black 

-.275 

.041 

-.174 

.000 

-.355 

-.195 

Latino 

-.123 

.038 

-.080 

.001 

-.197 

-.050 

Other 

.093 

.068 

.029 

.174 

-.041 

.227 

F/RL eligible 

-.177 

.034 

-.129 

.000 

-.243 

-.111 

Special Education 

-.315 

.062 

-.106 

.000 

-.438 

-.193 

LEP 

-.066 

.058 

-.025 

.251 

O 

OO 

.047 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = Confidence 
Interval. 

Table A29 

Linear regression predicting weighted overall GPA, POS completers versus CTE concentrators, 

South District 

Overall GPA 
n = 1,352 

Adjusted R 2 = 0.376 95% Cl 


Variable 

b 

SEb 

fi 

P 

lower 

upper 

(Constant) 

-.749 

.165 


.000 

-1.073 

-.424 

POS CTE Conc (1= POS Completer) 

.042 

.029 

.031 

.155 

-.016 

.099 

Male 

-.234 

.027 

■ 

'O 

o 

.000 

-.288 

-.181 

Asian 

.218 

.065 

.074 

.001 

.090 

.346 

Black 

.010 

.041 

.007 

.805 

-.071 

.091 

Latino 

-.043 

.036 

-.030 

.238 

-.113 

.028 

Other 

-.011 

.070 

-.003 

.876 

-.149 

.127 

F/RL eligible 

-.009 

.033 

-.007 

.783 

-.074 

.055 

Special Education 

-.002 

.060 

-.001 

.972 

-.120 

.116 

LEP 

.337 

.057 

.137 

.000 

.225 

.450 

Grade 8 pre-test math 

.008 

.001 

.417 

.000 

.007 

.009 

Grade 8 pre-test reading 

.004 

.000 

.257 

.000 

.003 

.005 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = Confidence 
Interval. 
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Table A30 


Linear regression predicting STEM credits earned, POS completers versus CTE concentrators, 
South District 


Variable 

Adjusted if 
b SEb 

STEM Credits 
n = 1,352 
= 0.305 
fi P 

95% Cl 
lower upper 

(Constant) 

-2.674 

.419 


.000 

-3.495 

-1.853 

POSCTEConc (1= POS Completer) 

OC 

OO 

.077 

-.055 

.015 

-.339 

-.036 

Male 

-.198 

.072 

-.064 

.006 

-.339 

-.058 

Asian 

.977 

All 

.133 

.000 

.638 

1.315 

Black 

.143 

.109 

.038 

.189 

-.071 

.357 

Latino 

-.015 

.095 

-.004 

.873 

-.202 

.171 

Other 

.107 

.186 

.013 

.566 

-.258 

.471 

F/RL eligible 

-.282 

.087 

-.087 

.001 

-.452 

-.112 

Special Education 

-.559 

.157 

-.083 

.000 

-.867 

-.250 

LEP 

.066 

.148 

.011 

.658 

-.225 

.357 

Grade 8 pre-test math 

.023 

.001 

.498 

.000 

.021 

.025 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = Confidence 
Interval. 

Table A31 


Linear regression predicting AP credits earned, POS completers versus CTE concentrators, 
South District 


Variable 

Adjusted R 
b SEb 

AP Credits 
n = 226 
= 0.388 
fi P 

95% Cl 

lower upper 

(Constant) 

-14.986 

.752 


.000 

-16.462 

-13.511 

POS CTE Conc (1= POS Completer) 

-.608 

.133 

-.098 

.000 

-.869 

-.346 

Male 

-.511 

.124 

-.090 

.000 

-.754 

-.268 

Asian 

2.043 

.297 

.151 

.000 

1.460 

2.625 

Black 

.374 

.188 

.054 

.047 

.005 

.742 

Latino 

.148 

.164 

.022 

.366 

-.174 

.470 

Other 

.030 

.321 

.002 

.926 

-.599 

.659 

F/RL eligible 

.019 

.149 

.003 

.899 

-.274 

.312 

Special Education 

-.097 

.273 

-.008 

.723 

-.633 

.439 

LEP 

.908 

.261 

.080 

.001 

.396 

1.419 

Grade 8 pre-test math 

.027 

.003 

.317 

.000 

.022 

.032 

Grade 8 pre-test reading 

.026 

.002 

.354 

.000 

.022 

.031 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = Confidence 
Interval. 
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Table A32 

Linear regression predicting college credits accrued, POS completers versus CTE concentrators, 

South District 

College Credits Accrued 


Variable 

Adjusted R 2 
b SEb 

n =1,352 
= 0.052 
fi P 

95% Cl 
lower upper 

(Constant) 

-.631 

.089 


.000 

-.806 

-.457 

POSCTEConc (1= POS Completer) 

-.008 

.016 

-.013 

.619 

-.039 

.023 

Male 

-.003 

.015 

-.005 

.854 

-.031 

.026 

Asian 

.098 

.035 

.076 

.005 

.029 

.167 

Black 

.038 

.022 

.058 

.084 

-.005 

.082 

Latino 

-.002 

.019 

-.003 

.923 

-.040 

.036 

Other 

.087 

.038 

.062 

.022 

.012 

.161 

F/RL eligible 

-.003 

.018 

-.006 

.856 

-.038 

.031 

Special Education 

.024 

.032 

.021 

.450 

-.039 

.088 

LEP 

.051 

.031 

.048 

.097 

-.009 

.112 

Grade 8 pre-test math 

.001 

.000 

.135 

.000 

.000 

.002 

Grade 8 pre-test reading 

.001 

.000 

.123 

.001 

.000 

.001 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = Confidence 
Interval. 

Table A33 


Linear regression predicting CTE GPA, POS completers versus CTE concentrators, South 
District 


Variable 

Adjusted R 
b SEb 

CTE GPA 
n = 1,352 
= 0.053 
fi P 

95% Cl 
lower upper 

(Constant) 

3.543 

.033 


.000 

3.477 

3.608 

POS CTE Conc (1= POS Completer) 

.073 

.030 

.064 

.016 

.013 

.132 

Male 

-.044 

.028 

-.042 

.117 

-.099 

.011 

Asian 

.032 

.068 

.013 

.641 

-.101 

.164 

Black 

-.187 

.042 

-.147 

.000 

-.270 

-.105 

Latino 

-.086 

.037 

-.071 

.022 

-.159 

-.012 

Other 

.016 

.073 

.006 

.826 

-All 

.159 

F/RL eligible 

-.086 

.034 

-.079 

.011 

-.152 

-.019 

Special Education 

-.250 

.060 

-.111 

.000 

-.368 

-.132 

LEP 

-.065 

.057 

-.032 

.250 

-.177 

.046 


Note. All coefficients rounded to two digits. LEP = Limited English Proficiency. Cl = Confidence 
Interval. 


217 



APPENDIX B: Qualitative Technical Appendix 


This appendix provides more detail on the qualitative methods employed in this study. In order 
to measure student outcomes in the intervention and comparison groups, we first needed to 
detennine whether or not the sites were implementing POS. Although some aspects of POS are 
relatively easy to identify — for example, the availability of credit transfer opportunities — other 
elements of POS as originally mandated by Perkins IV or as later fleshed out in OVAE’s Design 
Framework for POS (2010) may not be as easy to determine without direct observation or 
information from participating stakeholders. For that reason, we sought to differentiate, where 
possible, between POS as implemented at our intervention schools and “business as usual” CTE 
courses or programs offered at comparison high schools. During annual site visits to the three 
districts participating in the study, we used classroom observations and stakeholder interviews to 
tease out the differences between intervention POS and comparison CTE programs. Over the 
course of the study, as POS became more widely implemented for younger cohorts across 
districts and schools, the differences between the intervention condition and the counterfactual 
were becoming less obvious. 


Qualitative Data Collection 

Site Visits 

In order to create richly detailed case studies of the intervention and comparison high schools in 
the three districts participating in the study, we made annual visits to West and East Districts 
over a four-year period, beginning when our cohort students were ninth graders (the 2008-2009 
school year) and ending when they graduated from high school (the 2011-2012 school year). Due 
to the delayed entry of South District into the study, we were only able to make one site visit 
there during the 2011-2012 school year and thus have less qualitative data there than at the other 
two districts. 

To the extent possible, we tried to schedule our site visits at approximately the same time each 
year. The study’s two principal investigators conducted all of the visits. In West, with the 
exception of an initial two-day scouting visit, annual site visits typically spanned five school 
days, during which the two researchers each spent one day at the three POS high schools, then 
split up to each visit two comparison high schools. Comparison schools were chosen for 
potential visits if they had comparatively larger enrollments of sample students; we visited those 
schools where the principals agreed to host us (for a total of six schools over the course of the 
study). In East, site visits lasted between three to five school days. The two-member team spent 
one to two days at Blue Academy, then divided up the four comparison schools for one-day 
visits, alternating comparison schools each year (e.g., Researcher 1 visited Emerald and Indigo in 
2009-2010 and Heliotrope and Neon in 2010-2011, while Researcher 2 visited Heliotrope and 
Neon in 2009-2010 and Emerald and Indigo in 2010-2011). In the case of South District, which 
was the last district to join the study, we collected historical systems data for our cohort’s full 
four years of high school and made one five-day site visit to six high schools during our cohort’s 
senior year (2011-2012). The research team visited four schools together and two schools 
separately in South District. 
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Across years, a typical school visit usually included an interview with the school’s principal (or, 
if she or he was not available, an assistant principal for curriculum and instruction), an interview 
with a guidance counselor, an interview with the school’s CTE program coordinator or academy 
coordinator (if any), one or two student interviews, and two to three classroom observations. We 
found it easier to create well-balanced schedules in schools using periods (typically 7 or 8 
periods in a day) versus those following block schedules. In the case of the latter, in order to fit 
in an adequate number of observations and interviews, we sometimes conducted classroom 
observations during the first half of a block class and used the second half for interviews. Lunch 
periods were also used for interviews with students and staff. 

Interviews 

We conducted both individual and group interviews (Krueger, 1994) with key stakeholders (i.e., 
high school students, teachers, guidance counselors, and administrators, including principals, 
assistant principals, and program or academy coordinators) at all schools visited. Interviews were 
also conducted with state and district personnel, business and industry partners, and 
postsecondary personnel. Semi-structured interview protocols were used in all settings; these 
were updated as needed for stakeholders who were interviewed more than once. Adult interviews 
usually lasted 45 minutes to 1 hour; student interviews generally ranged from 20 to 30 minutes. 
We did not attempt to achieve parity of representation of all types of stakeholders at the 
individual school level, opting instead to capture the perspectives of a broad range of 
stakeholders across sites. As shown in Table Bl, we conducted 95 interviews in West District, 80 
in East District, and 44 in South District (where we were able to conduct only one site visit) for a 
total of 2 19 interviews. In those interviews, we spoke with a total of 235 POS stakeholders of 
various kinds. 

Whenever possible, we asked the same or similar questions of each group of stakeholders so that 
data could be triangulated. Notable examples include questions about the four mandated 
elements of POS, like the availability of credit transfer opportunities or the alignment of 
secondary and postsecondary education. In later years of the study, interview protocols were 
expanded to capture the presence of the ten components of OVAE’s POS Design Framework 
(2010). Some questions were only posed to specific groups of stakeholders. For example, CTE 
teachers, administrators, and guidance counselors were asked about (a) their involvement in the 
implementation of POS (e.g., whether they participated in state- or district-led trainings on POS), 
(b) how career guidance was integrated into programs, (c) the climate and culture of the school, 
(d) parent involvement, and (e) their vision for the future of their students. Teachers were asked 
about the integration of academic and CTE curricula and whether they had opportunities to 
collaborate with other teachers to plan lessons. Teachers and principals were asked how they 
used assessment data — both academic and technical — to enhance instruction and improve 
student outcomes. Teachers and administrators in both secondary and postsecondary settings 
were asked about the participation of business and industry partners in CTE programs. In West 
District, we interviewed two of those business and industry partners. Teachers and students were 
asked to describe the types of assessments used in their academic and CTE classes. Students 
were asked, as applicable, how they had chosen their POS in high school, about their college and 
career plans, and whether they thought their friends would graduate from high school. District 
and state personnel were asked to describe their involvement in the development and 
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implementation of POS. Postsecondary personnel were asked about the history of alignment 
efforts between the district and their institution. 7 

Table B1 


Interviews conducted at study sites, by district 


West 

5 visits 

Intervention Comparison 

3 schools 6 schools 

East 

4 visits 

Intervention Comparison 

1 school 4 schools 

South 

1 visit 
n/a 

6 schools 

High school 








Administrators 

7 


9 

2 


13 

13 

Counselors 

7 


2 

3 


4 

6 

Teachers (career) 

25 


7 

10 


11 

0 

Teachers (academic) 

3 


0 

2 


2 

0 

Students 

15 


5 

7 


13 

25 

School-level totals 

57 


23 

24 


43 

44 

District personnel 


5 



2 


0 

Business partners 


2 



0 


0 

State dept, of education 


1 



0 


0 

Community college 








Administrators 


3 



5 


0 

Counselors 


1 



1 


0 

Instructors (career) 


3 



6 


0 

TOTAL interviews 


95 



80 


44 

TOTAL interview 


103 



83 


49 

participants 









Note. The total number of interview participants does not equal the total number of interviews because (a) some 
people (i.e., principals) were interviewed more than once over the course of the study, and (b) some interviews 
included several people (i.e., an interview with a principal in which the career academy coordinator was also present 
and contributing to the conversation). 

The interview selection process was both purposive and incidental, largely mirroring the 
development of the site visit schedule. In general, teachers whose classes were being observed 
that day, or who taught in the same program as the classes being observed, were invited to 
participate in interviews during their lunch breaks or planning periods. We asked to interview 
students from the same classes or programs. In all cases, we could only interview those students 
whose parents had previously returned signed consent forms allowing them to participate. 

In 2011-2012, in order to ensure a range of student perspectives, we identified potential 
interviewees using our definition of at risk (i.e., students who scored at or below the 20th 
percentile on eighth-grade baseline assessments of math, reading, or science, and who were firee- 
lunch eligible), and we asked the districts’ research offices to submit the list to the schools in 
order to send letters and consent forms home to students in this group. Schools complied, and we 
selected interviewees from those students whose parents returned the forms. 


7 Interview protocols are available from the authors upon request. 
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Annual site visits also included interviews with district officials (i.e., administrators from the 
district CTE offices), business and industry partners, the local community college (including 
instructors, administrators, and admissions/enrollment counselors), and the state department of 
education. Most of these interviews took place after the end of the school day. 

Classroom Observations 

We observed POS classes at the intervention schools and CTE classes at the comparison schools. 
We also observed some academic classes — both core requirements and classes that might be 
related to a POS (i.e., Epidemiology) — but these were not a major focus of our work. Indeed, in 
West District, we observed no academic classes in the comparison schools because we wanted to 
focus our limited time and resources on understanding CTE in that large district. 

Classroom observations ranged from 45-90 minutes, depending on the duration of the class 
periods or blocks at the school. Like the interview selection process, our selection of classes for 
observation was both purposive and incidental; schools’ schedules and the availability and 
willingness of teachers to be observed defined our choices. Each year, we sought to observe only 
those classes in which our cohort was enrolled; in the case of CTE classes, we were able to use 
school or district four-year POS course sequences to identify appropriate classes (e.g., 
introductory CTE classes in Grades 9 and 10; advanced or capstone CTE classes in Grades 1 1 
and 12). In choosing CTE courses, we attempted to observe a broad range of programs, taking 
care to “match,” where possible, POS courses in intervention schools with their closest 
analogues in comparison high schools (e.g., during the same site visit, we observed Introduction 
to Culinary Arts in a culinary POS and a Foods and Nutrition I class in a comparison school 
offering family and consumer science courses). In the case of academic classes, we chose to 
focus on one core academic area each year (e.g., English in Grade 9, math in Grade 10, science 
in Grade 1 1) across intervention and comparison schools, in order to get a stronger sense of how 
these academic subjects were presented in POS versus non-POS settings. All teachers were asked 
in advance if they were willing to have their classrooms observed and assured that these 
observations were non-evaluative. Nearly all teachers were amenable to being observed, and 
many expressed interest in the goals and outcomes of the study. 

Our classroom observation protocol included three activities, two completed during the class 
(marking a time interval fonn; taking running notes) and one completed later (filling out a 
detailed classroom observation form using the interval fonn and running notes as guides). The 
full observation protocol spanned the range of low-inference to ethnographic observations (cf. 
Castellano & Datnow, 2004; Datnow, Borman, Stringfield, Rachuba, & Castellano, 2003). Our 
goal was to capture the full scope and richness of the classroom experience for teachers and 
students, and the same observation forms were used in both academic and CTE classes. The time 
interval form was used to generate an overall snapshot of classroom activities. The classroom 
observation fonn included questions intended to capture elements of POS (both the four 
mandated elements and the ten components described by OVAE) as a means of generating 
comparative data on the intervention versus comparison conditions. This latter form is found in 
Appendix C. Similar combinations of data gathering instruments have been used successfully in 
studies of school effectiveness and refonn design implementation (Castellano & Datnow, 2004; 
Datnow et ah, 2003; Stringfield et ah, 1997; Teddlie & Stringfield, 1993). 
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The first instrument, the time interval form, is a modified version of the Classroom Observation 
Measure (COM), developed and validated at the University of Memphis (Ross et ah, 1994). It 
includes one section that requires the observer to take systematic observations of the entire class 
at regular intervals (timed according to the duration of the class, but generally running between 
6-10 minutes). This process yields low-inference data points regarding classroom orientation 
(e.g., teacher-led, project-based, individual student vs. group work). At the end of the class, 
observers complete a second, more global section of the form that gauges the extent of various 
classroom activities (e.g., the amount of direct instruction, independent student work, use of 
computers or other technology). The purpose of the time interval form is to reduce certain 
classroom events to variables that can then be compared across classes. 

Figure B 1 depicts a pseudonymous sample time interval form. At the top of the form, the 
observer enters general infonnation about the classroom observation, including the date, school, 
class, teacher, and grade level of the students present. Based on length of the class observed, the 
researcher also enters the specific times that correspond with each interval segment. For 
example, Figure B1 instructs the researcher to record observations in six-minute intervals. Given 
that the sheet provides for ten data points, this sample was for a 54-minute class period. 
Observations begin and end at the bell in order to capture whether teachers are using the full 
class period or allowing late starts or early wrap-ups. Every six minutes, the researcher makes a 
snapshot observation of the orientation of the class and marks all relevant items in the column for 
that interval in Section A. Section A allows the researcher to capture a range of activities and 
student orientations for each interval. For example, at the third time point in Figure B1 (8:12 
a.m.), the majority of the class was receiving teacher-led direct instruction while one student 
worked independently at a computer workstation and two students engaged in off-topic chatter at 
the back of the room. In this case, the observer heavily circled “teacher led” (indicating that 40% 
or more of the students were engaged in this activity) and lightly circled both “independent” and 
“off task” (indicating that less than 10% of the students were engaged in these activities). 

Overall, Section A of this sample form shows that in this class, students were still transitioning 
into the class when the first bell rang. The teacher then lectured; after this lecture, students 
worked in small groups and then led the class until the final bell rang. A minor amount of off- 
task behavior occurred. What Section A does not reveal is the focus of the teacher’s lecture, the 
nature of the students’ small-group work, or the content of the student-led portion of the class, 
among other pertinent details. That kind of information is contained in the observer’s running 
notes and distilled in responses to the questions contained in the classroom observation fonn. 

Section B of the time interval fonn is used to record the amount of time expended on various 
pedagogical practices and activities (e.g., sustained writing, use of alternative assessments, use of 
computers or other technology, dealing with discipline problems) during the class, where None = 
no class time, A little = less than or equal to 10% of class time, Some = 11% to 39% of class 
time, and Extensive = 40% or more of class time. The sample presented in Figure B1 shows that 
computers were used extensively during the class period, with the classroom’s projection system 
used somewhat less. In this sample, the observer indicated that activities were neither 
challenging nor hands-on. Students engaged in some dialogue with each other and the teacher, 
and the teacher offered a little feedback but did not act as a coach or a facilitator. 


222 



Figure B 1 

Sample time internal form (pseudonymous) 


Rigorous Tests of Student Out tomes in CTE Programs of Study 
Observation Time Intervals 

Record your observation; in sir- minute intervals. One minute alter the bell rings, take the next minute to observe 
the classroom activity and record yo«r observation (circle H>. Do this again six minute; later and so on for the 
length of the class. 


Segment Actual Time 


1 

8:00 

2 

8:06 

3 

8:12 

4 

8:18 

5 

8:24 

6 

8:30 

7 

8:36 

8 

8:42 

9 

8:48 

BeD 

8:54 


School name: 

Pi! -U a 

Teacher name: 

John 0. Public 

Class title: 

Intro to Information Technology 

wmnmwm 

9““ grade 

Date: 

October 19. 2009 

Time: 

8:00-8:54 am. 

Observer: 

Smith 


Men WWI ,! Iff! 1 J W W m HWfl ffl 4 ff. WWW 




*1Wv>W 


Teacher led 

1 < 


49k 


> 5 

6 

7 

8 

9^ 

Bell 

Team teaching 

1 

7 


7 

5 

6 

7 

8 

9 

Bell 

Independent 

1 

Z_ 2 J 

: 3 ; 

4 

5 

6 

7 

8 

9 

Bell 

Small group 

1 

2 

3 

4 i 

H 5 12 

C6^ 

rn 

1 8 

9 

Bell 

Pairs 

1 

2 

3 

4 

“5" 



8 

9 

BeU 

Student led 

1 

2 

3 

4 

5 

6 

7 t 

:8 j 

9 3 

mm* 

Media led 

1 

2 

3 

4 

5 

6 

7 

8 

" 9 " 


Project-based 

1 

2 

3 

4 

5 

6 

7 

8 

9 


Transition activities 1 

m 

■ 2 

3 

4 

5 

6 

7 

8 

9 


Off-task 

i 

2 

LO 

L aJ 

5 


■ ' 

8 

9 



B. Overall Observation 



None 

A Little 

Some 

Extensive 


0 

< 10% 

11-39% 

>40% 

Direc t instmc tionbyteacher 

N 

L 

C*S 

E 

Independent work by students 

N 

CL2 

s 

E 

Teacher provided feedback (answers, information, etc .) 

N 

TT3- 

s 

E 

Students engaged irrelevant dialogue with each other or with teacher 

N 

L 

C 

E 

Sustained writing 

CN 2 

L 

s 

E 

Computer used as tool or resource 

FT 

L 

s 

C E 3 

Other technologyuse das tool or re source (which?) prcytcSon system 

N 

L 

C_s_3 

E 

Experiential, hands-on learning 

t N ^ 

L 

s 

E 

Challenging activities 

CN 5 

L 

s 

E 

Alternative assessment strategies 

0 J 

L 

s 

E 

Teacher acted as coach/facilitator 

cn.5 

L 

s 

E 

Discipline problems 

N 


s 

E 
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In our analyses of the data collected in these time interval forms, means were calculated for the 
presence of various classroom orientations (Section A) and specific pedagogical practices 
(Section B). Means were calculated by district to determine the overall percent of all 
observations for each of the orientations and practices. Within each district, we then compared 
the means of career to academic classes as well as overall intervention to comparison classes. 
The second instrument, the classroom observation form (see Appendix C), was used to generate 
rich descriptions of observed classes. The form asked observers to summarize the classroom 
setting — including the number of students present (including gender, race/ethnicity, and grade 
level), the experience and demeanor of the teacher, the condition of the room and its technology 
and equipment, and the topic of the class — and to assess the presence or absence of various 
elements of POS and other classroom practices. Questions were derived in part from the four 
Perkins-mandated elements of POS, the ten supporting components contained in the policy 
guidance framework, extant research on CTE classroom practices (as reviewed in Castellano et 
ah, 2003), and more general research on effective teaching and learning practices (Newmann & 
Wehlage, 1995; Tharp, 1997). 

Questions related to POS included evidence of integrated curricula (e.g., a career focus in an 
academic classroom, applied academics in a CTE classroom, evidence of team teaching), dual 
enrollment opportunities, business partnerships, planned or impromptu career guidance, work- 
based learning opportunities, and secondary-postsecondary alignment. The form also included a 
section on soft skills, also known as the SCANS skills (Secretary’s Commission on Achieving 
Necessary Skills, 1991) or employability skills, which have been identified as necessary for 
success in the workplace. Observers noted whether or not these skills were attended to in the 
observed class. 

The sections of the form that elicit responses on effective teaching and learning were based on 
similar fonns from previous studies (Castellano & Datnow, 2004; Datnow & Yonezawa, 2004) 
that were modified for this study. The form also incorporated Tharp’s (1997) principles for 
effective classroom instruction: 

1 . facilitating learning through joint productive activity between teachers and students; 

2. developing competence in the language and literacy of instruction throughout all 
instructional activities; 

3. contextualizing teaching and curriculum in the experiences and skills of home and 
community; 

4. challenging students toward cognitive complexity; and 

5. engaging students through dialogue. 

The classroom observation form included Newmann and Wehlage’s (1995) tenets for successful 
school restructuring: involving students in higher order thinking and substantive conversational 
exchange, producing complex understandings, and helping students connect substantive, aca- 
demic knowledge with current events, personal experience, or background. 

Finally, the form sought to assess the general culture and climate of the classroom and capture 
any unusual or noteworthy incidents or activities not otherwise included in the fonn. Observers 
were encouraged to reflect on their overall impressions of the class. 
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Some of the questions on the classroom observation form supplemented the measures of the time 
interval form. For example, the time interval form showed how often computers or other 
technology were used during the class, but did not elicit how or why they were used. The 
classroom observation form asked observers to provide explicit detail on the number of 
computers or other technology in the room, how those computers were incorporated into 
instruction, and how expertly and efficiently students used them. 

We conducted a total of 125 classroom observations across all districts. Of these, 117 time 
interval forms and 115 classroom observation forms were analyzable. Omitted from the 
analyzable totals were (a) elective classes (e.g., art, theater, ROTC) and (b) classes for which 
either one or the other observation form was not completed (in a few cases, circumstances 
prevented the completion of forms, although in all cases, running notes were taken). Table B2 
summarizes the total number of observations conducted by district and the type of class, minus 
electives and those observations for which forms were missing, yielding the total analyzable 
number of observation forms. 

Table B2 


Number of observations conducted and forms completed, by district and class type 


Observations/Forms Completed 

Total 

Observed 

N 

Career 

Classes 

N 

Academic 

Classes 

N 

Elective 

Classes 

N 

Classes 

Omitted 

from 

Sample 3 

N 

Analytical 

Sample 

N 

West District Intervention 

Total observations conducted 

46 

35 

10 

1 

1 


Timed interval form 

46 

35 

10 

1 

1 

45 

Classroom observation fomi 

46 

35 

9 

1 

1 

44 

West District Comparison 

Total observations conducted 

16 

16 

0 

0 

0 


Timed interval form 

16 

16 

0 

0 

0 

16 

Classroom observation fomi 

15 

15 

0 

0 

0 

15 

East District Intervention 

Total observations conducted 

16 

13 

3 

0 

0 


Timed interval form 

16 

13 

3 

0 

0 

16 

Classroom observation fomi 

16 

13 

3 

0 

0 

16 

East District Comparison 

Total observations conducted 

26 

22 

3 

1 

1 


Timed interval form 

23 

19 

3 

1 

4 b 

22 

Classroom observation fomi 

23 

19 

3 

1 

4 b 

22 

South District 

Total observations conducted 

21 

17 

3 

1 

1 


Timed interval form 

19 

15 

3 

1 

3 C 

18 

Classroom observation fomi 

19 

15 

3 

1 

3 C 

18 


Notes . a Omitted classes were either electives or classes for which the observation forms were not completed. 
b Forms not completed for 3 career courses. The elective class is the fourth omitted observation. 
c Forms not completed for 2 career courses. The elective class is the third omitted observation. 
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Reliability 


We conducted several reliability checks of the classroom observation instruments and our 
interview transcript coding procedures. 

Observation Instruments 

Two observation instruments were used: a time interval fonn and a classroom observation form 
(cf. Appendix C). We conducted reliability exercises on these forms at two different time periods 
and at two different schools. The first check took place during the site selection phase and 
involved observations of a graphic design course and a child development course at a high 
school that had been considered for participation but was not ultimately selected for inclusion in 
the study. The second reliability check occurred in West District, prior to beginning the formal 
data collection process, during a visit to finalize the details of West’s participation. The classes 
we observed for this second round of reliability checking included a 3D animation class, a 
graphic design class, and two ninth-grade soft skills classes. After completing the classroom 
observations, we collected the research instruments and tallied the responses. 

For the time interval forms, we used Cohen’s kappa statistic (1960) to measure inter-rater 
reliability. This statistic is considered more robust than a simple percentage agreement because it 
takes into account the probability of the agreement occurring by chance. On the first section of 
the time interval form, which asks the observer to choose among such classroom orientations as 
teacher led or project-based at each interval, the research team reached 7 1 % agreement, which is 
in the “substantial agreement” range (Landis & Koch, 1977). On the second section of the 
instrument, which asks the extent to which various practices were observed, such as sustained 
writing or computer used as tool or resource, the team scored 65%, also within the “substantial 
agreement” range. There has been some criticism of Landis and Koch’s guidelines for what 
constitutes substantial agreement, and indeed, in some circles, the kappa statistic itself has been 
questioned (cf. Maclure & Willett, 1987). As an alternative, the simple agreement percentages 
between the two observers on the time interval fonns were 92% for classroom orientation and 
86% for prevalence of practices observed. 

The classroom observation form is an open-ended fonn that solicits more richly detailed 
information about the classroom (or shop or lab), such as the resources and technology available 
for use (and in use on that day), what teachers and students did during the observation, and 
whether specific academic or career education elements were evident that day (e.g., whether 
students did any math, engaged in activities requiring higher order thinking skills, or produced a 
product with value outside of the classroom). In detennining inter-rater agreement for this form, 
the criterion for each question on the form was: Is it evident that the observers were seeing the 
same class? For example, in the case of one of the freshman studies classes observed, for the 
question, Was there evidence today of lesson planning and time management on the part of the 
teacher, both observers captured one teacher’s poor use of time, especially at the end of class. 
One observer wrote: 

Not enough. It seemed as though she had not planned what she would do after that point. 

She began a half-hearted discussion about answers to the questions, and what is a short- 
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versus a long-term goal. But some students were still working. At 15 minutes to go, she 
wrapped up the “discussion” and told them to keep working. If they finished this 
assignment, she said, “find something quiet to do.” 

The other observer noted: 

Time was not well-used in this class. The reading and seatwork did not fill the whole 
class, so the teacher told the students 15 minutes before the end of the period that, if they 
were finished with their work, they should take out a book and be quiet so that other 
students could finish. By this point, a lot of chatting and disruption had broken out. 

We tallied all responses that were and were not in agreement across observers, resulting in an 
88% agreement rate across all four observations. Two areas in which the coders diverged are 
worth noting. First, the question with the highest disagreement across coders asks observers to 
provide a count of the number, race/ethnicity, and gender of the students in the class (e.g., 26 
students: 13 males - 8 Black, 2 Latino, 1 Asian, 2 White; 13 females - 6 Black, 3 Latina, 2 
Asian, 2 White). Depending on where an observer is seated in the classroom, it can be difficult to 
count and classify students according to their gender and race/ethnicity without attracting undue 
attention during class (e.g., standing to see students in a far corner). In addition, in today’s urban 
districts, race/ethnicity and even gender are not always obvious to an observer. 

The second area of divergence was the SCANS skills-based question, Were students responsible 
for selecting the appropriate tools or equipment to complete a task ? In different observations, 
each observer answered “yes” based on a very broad interpretation of tools and equipment: “Yes, 
in the sense that they were selecting from among the self-motivation techniques the ones that 
were best for them,” and “Yes, the students chose the appropriate tools to work with within their 
3D animation software and design programs. These were ‘virtual’ tools, but tools nonetheless.” 
Neither observer was consistent across reliability observations in the breadth of their 
conceptualization of tools and equipment. It was useful to identify these areas of observer 
divergence, but the areas the coders held in common were far greater — giving confidence that 
the classroom observation form was capturing the classroom experience in reliable ways. 

Interview Transcript Coding 

To establish inter-rater reliability for the study’s interview transcript coding process, the two 
study team members each coded five interview transcripts: one each for a student, a counselor, a 
teacher, a principal, and a business partner. These coded transcripts were then compared, code by 
code. If a code was found next to an interviewee comment on one researcher’s transcript, it was 
sought on the other transcript in the same place. The number of agreements across coded 
transcripts was tallied and a percentage produced. Only coder agreement was being tallied, not 
the appropriateness of the code for any particular comment in the transcript. This method of 
establishing agreement means that in some cases, some interview data that should have been 
coded may not have been coded. However, if we had coded every mention of every topic, the 
reports created in our chosen qualitative data analysis software package, HyperResearch, would 
have been too large and unwieldy for axial coding analyses; such reports would have contained 
extraneous data. Instead, we created coding guidelines, such as whether coding an instance of a 
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topic would further our knowledge of that concept. We depended on the researcher’s eye and 
judgment to distinguish and code content-rich data, not extraneous data. 

For example, the following section of a transcript was initially coded as postsecondary 
partnerships, but when the researchers reviewed the coding for reliability, it was agreed that no 
new infonnation appears in this comment that could help us understand the specific 
postsecondary partnerships at this site, and thus the code was not warranted. 

Principal: Those courses, you know, are going to lead to certification. They’re going to 
lead to college credit because of the relationship between us and the Rochester Institute 
of Technology, Project Lead the Way. They can get college credits for scoring at a 
certain level on those courses, on the exams, and so those kids kind of have to apply for 
that. 

This site’s partnership with the Rochester Institute of Technology is mediated through Project 
Lead the Way (PLTW). High schools across the country that participate in PLTW share this 
same partnership. As such, although this excerpt is nominally about a postsecondary partnership, 
it does not tell us about the school’s local partnerships that yield opportunities for high school 
students to visit the college campus, take classes, and begin to see postsecondary education as 
their natural next step. 

The inter-rater reliability results for these transcripts ranged from 30% to 58% agreement. In 
reviewing the results, one reason for the low agreement was that some of the codes, although not 
duplicative, could be used in pairs or groups. A good example is the various codes related to 
assessment: We coded instances of interviewee comments on state assessment, academic 
assessment, and technical assessment. The codes were based on some of the interview questions 
and the types of assessment between which we wanted to differentiate. However, in some 
conditions, state assessment was double-coded with either academic assessment or technical 
assessment. Sometimes one coder would double code but the other coder would not, lowering 
our agreement ratio. Other examples of such code pairs are dual credit and postsecondary 
partnerships, work-based learning and business partnerships , and common planning time and 
teacher collaboration . In some contexts, either one or the other of the codes in these pairs was 
used and inter-rater agreement was high. In other contexts, the codes were used in the pairs or 
groupings, and agreement was still high. But in certain contexts, only one coder used the code 
groupings, effectively lowering the agreement ratio in the inter-rater reliability exercises. These 
code pairs/groupings were identified and queries of the HyperResearch database took them into 
account, usually by querying both codes to ensure all relevant infonnation was included in the 
code report. 

The writing process for this report provided a means by which to overcome this low inter-rater 
agreement ratio. Given that the two co-principal investigators conducted all of the study’s site 
visits and coded all of its transcripts, these researchers became very familiar with their own 
corpus of interviews and were able to bring this familiarity to their readings of each other’s 
drafts. As an example, if a specific point was made by one of the researchers in a draft section of 
the report, the other researcher drew on her experience of the interview data in order to provide 
any needed confirmatory (or tempering) examples not used by the first author. Given the size of 
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the qualitative database, it was more often the case that we had too many examples to illustrate a 
point than too few. Again, collaboration helped detennine which examples were finally used in 
the report. 

Transcript coding was a mammoth task that had to be delimited in a manageable way. Although 
we did not achieve high inter-rater reliability, we believe that we captured the major themes in 
the transcripts and identified and corrected for any major discrepancies when we queried and 
used the data. 
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APPENDIX C: Classroom Observation Form 


Part I: General Classroom/Lab/Shop 
Ila. The Students (# students, race, gender, grade level) 
lib. Are any of the students in a program nontraditional for their gender? 

12. The Teacher (approx, age or experience level, race, gender, demeanor, level of engagement) 

13. Class Description Briefly, what was this class about? What was the subject? What was the 
topic for today? 

I4a. Facilities, Resources, & Climate (Is there adequate space for everyone? Lighting? Books? 
Equipment? Work stations? How is the state of repair/disrepair?) 

I4b. Describe and comment on the integration of technology in instruction. 

I4c. Describe the technology in the classroom/shop/lab. Does it appear to be state of the art? 
Describe how the students were interacting with the technology (the extent to which it was used 
at all, the level of student engagement with the technology, etc.). If possible, ask the teacher if 
the technology is adequate for the purpose of the class. 

Part II: Programs of Study (POS) 


II. 1. Integrated Curriculum 

Ilia. Did the teacher incorporate concepts from both academic and CTE topics? Describe. (For 
example, did a math teacher use an example from an academy topic, did the computer teacher 
say part of the students’ grade on an assignment/project would be based on their spelling and 
punctuation, etc.) 

Illb. Did you see any evidence today of teachers teaming across classes or team teaching to 
integrate CTE and academic curricula? Describe. 

11.2. Work Based Learning (WBL) 

II2a. Were student work experiences used for academic activities or assignments? Do you know 
if this work experience was connected to the class? 

II2b. Did students bring up their WBL experiences (i.e., internships)? If so, what did they say? 

11.3. Assessment 

II3a. Did you see any assessment going on in the class? If so, describe. Did you see the use of 
portfolios, demonstrations, or exhibits? 


230 



II3b. Did the assessments focus on academic AND CTE competencies? Example? 

II. 4. Other POS Elements 

II4a. Did you see any evidence today of articulation with postsecondary 
programs/standards/curricula? (in discussion, on posters, anything) 

II4b. Did you see any evidence today of business partnerships? Describe. 

II4c. Did you see any evidence today of dual enrollment opportunities? Describe. 

II4d. Did the teacher incorporate planned or impromptu career guidance or advice today? 
Describe. 

II4e. Were the standards for the course (CTE or academic) posted anywhere? 

II4f. Did you see any evidence today of career technical student organizations (CTSOs)? 
Describe. 

II4g. Were “all aspects of an industry” part of the class discussion or activity? 

II4h. Was the local labor market the focus of class discussion/activities? 

Part III: SCANS Skills 
7//.1. SCANS Workplace Competencies 

Illla. Did students work as teams on projects or activities? (This is more than just working in 
groups to answer questions at the end of the chapter.) If so, look for and describe the following: 

• Were the teams representative of the diversity of the class? 

• Did students appear to work well together? 

• Were students teaching other students? 

• Were students negotiating solutions to problems? 

• Were students leading the team toward completion of team goals? 

• Did most students contribute, act responsibly during the activity? 

• Describe the activity requiring team work 

Illlb. Did the activity require the management of time, money, materials, or human resources? 
Was there any evidence that students took any of this into account in their work? 

IIIlc. In what ways did the students work with information? (Discuss all that apply: acquire, 
evaluate, organize, maintain, interpret, or communicate information, use computers to process 
information.) Or did the teacher/textbook just supply or transmit infonnation that the students 
simply reproduced (e.g., textbook-tied worksheets, etc.)? 
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Illld. Were students responsible for selecting the appropriate tools or equipment to complete a 
task? Were they responsible for maintaining or troubleshooting these tools? 


III.2. SCANS Basic Skills, Thinking Skills 

III2a. Did you observe any student reading? Describe. 

III2b. Did you observe any student writing? Describe. 

III2c. Did you observe any arithmetic or mathematics? Describe. 

III2d. Did students participate in discussions or presentations? Describe. 

III2e. Did students teach or lead a portion of the class? Describe. 

III2f. Was decision-making part of any class activities? Describe. 

Part IV: Restructured Teaching and Learning (cf. Newmann & Wehlage) 

IV1. Were students conducting disciplined inquiry (e.g., using ideas/concepts to interpret, 
explain, or solve specific concrete infonnation or activities. Note if this instance includes 
students providing coherent accounts of their understanding in elaborated written 
communications.)? Explain how you know this. 

IV2. Were students engaging in higher order thinking skills? Describe. 

IV3. Was the focus of the class ever on issues connected to the local community? 

IV4. Were students engaged in producing a “product” that would have value or use beyond the 
classroom? Explain how you kn ow this. 

Part V: General Classroom Culture 


VI. Pedagogy 

Via. Who initiated teacher-student interaction? Was it about clarifications and known- answer 
questions, or higher-level? 

Vlb. Did the teacher use multiple instructional strategies? 

Vic. Was there evidence today of lesson planning and time management on the part of the 
teacher? 

Vld. Did students appear to understand what they were learning? Did they have the appropriate 
background knowledge/scaffolding? 
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V 2. Expectations 

V2a. Describe the explicit and implicit expectations for this class? Did the expectations seem to 
be equally demanding for all students? 

V2b. Was there evidence that students were being held to specific academic or technical 
standards? 


Part VI: Brief Descriptions 
VI. 1. Miscellaneous Classroom Activities 

Please use the space below to note any classroom activities that were not captured by the above 
questions (i.e., options for students who are behind, mention of an individual graduation plan). 

VI. 2. Reflections 

Reflect on the classroom note your overall perceptions. Anything unusual, disturbing, dull, or 
unique? 
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APPENDIX D: Senior Exit Survey Respondent Characteristics 

In order to take into account any potential bias with respect to the senior survey, crosstabs were 
run on any baseline differences between survey respondents and nonrespondents as well as 
differences between intervention respondents and comparison respondents. The tables follow, 
first for West District and then East District. Senior exit surveys were not conducted at South 
District. 

Table D1 


Background characteristics of senior survey takers and nontakers, by intervention status, West 
District 



Intervention 

Comparison 


Takers 

Nontakers 

Takers 

Nontakers 


(A =456) 

ll 

(A =325) 

o 

in 

II 

Characteristics 

(%) 

(%) 

(%) 

(%) 

Gender 





Male 

45.6 

52.0* 

34.8 

30.8 

Female 

54.4 

48.0 

65.2 

69.2 

Race/Ethnicity 





Black 

11.8 

10.0 

5.8 

10.1 

White 

41.9 

40.3 

40.9 

38.9 

Latino 

32.5 

32.4 

35.4 

34.7 

Asian 

12.5 

16.8 

16.6 

15.7 

Other 

1.3 

0.4 

1.2 

0.6 

Free-lunch eligible 

27.6 

27.3 

26.2 

29.4 

Limited English Proficient (LEP) 

1.1 

2.9* 

2.8 

2.8 

Special Education 

2.0 

3.3 

5.2 

6.3 


Mean 

Mean 

Mean 

Mean 


(SD) 

(SD) 

(SD) 

(SD) 


(n = 428) 

(n = 663) 

(n = 320) 

(n = 496) 

Grade 8 pre-test math scale score 

367.12 

369.11 

374.23 

365.97 


(73.10) 

(77.60) 

(81.14) 

(80.34) 

Grade 8 pre-test reading scale score 

345.15 

343.56 

351.37 

344.71 


(53.30) 

(54.71) 

(59.30) 

(56.34) 


* p< .05. 
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Table D2 

Background characteristics of students who took senior survey, by intervention status, West 
District 



Took Survey 


Intervention 

Comparison 


(A =456) 

(A =325) 

Characteristics 

(%) 

(%) 

Gender 



Male 

45.6 

34.8** 

Female 

54.4 

65.2 

Race/Ethnicity 



Black 

11.8 

5.8** 

White 

41.9 

40.9 

Latino 

32.5 

35.4 

Asian 

12.5 

16.6 

Other 

1.3 

1.2 

Free-lunch eligible 

27.6 

26.2 

Limited English Proficient (LEP) 

1.1 

2.8 

Special Education 

2.0 

5.2* 


Mean 

Mean 


(SD) 

(SD) 


(n = 428) 

(n = 320) 

Grade 8 pre-test math scale score 

367.12 

374.23 


(73.10) 

(81.14) 

Grade 8 pre-test reading scale score 

345.15 

351.37 


(53.30) 

(59.30) 


* p < .05. ** p < .01. 
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Table D3 


Background characteristics of senior survey takers and nontakers, by intervention status, East 
District 



Intervention 

Comparison 


Takers 

Nontakers 

Takers 

Nontakers 


(N = 158) 

(N= 218) 

(A =242) 

ll 

o 

Characteristics 

(%) 

(%) 

(%) 

(%) 

Gender 





Male 

53.2 

51.4 

47.9 

50.8 

Female 

46.8 

48.6 

52.1 

49.2 

Race/Ethnicity 





Black 

75.3 

67.9 

76.4 

* 

67.6 

White 

8.2 

12.4 

5.8 

** 

13.7 

Latino 

8.9 

11.5 

7.4 

12.2 

Asian 

3.8 

3.7 

6.2 

_ . ** 

2.4 

Other 

3.8 

4.6 

4.1 

4.1 

Free-lunch eligible 

65.8 

70.2 

67.4 

65.9 

Limited English Proficient (LEP) 

2.5 

5.5 

1.7 

6.3" 

Special Education 

1.9 

2.8 

0.8 

2.9 


Mean 

Mean 

Mean 

Mean 


(SD) 

(SD) 

(SD) 

(SD) 

Grade 8 pre-test math scale score 

363.82 

362.78 

363.45 

362.63 


(6.83) 

(7.03) 

(7.22) 

(7.68) 

Grade 8 pre-test reading scale score 

360.05 

359.80 

359.77 

359.65 


(7.10) 

(7.12) 

(6.65) 

(7.87) 


* p < .05. ** p < .01. 
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Table D4 

Background characteristics of those who took senior survey, by intervention status, East District 



Took Survey 


Intervention 

Comparison 


(N= 158) 

(A =242) 

Characteristics 

(%) 

(%) 

Gender 



Male 

53.2 

47.9 

Female 

46.8 

52.1 

Race/Ethnicity 



Black 

75.3 

76.4 

White 

8.2 

5.8 

Latino 

8.9 

7.4 

Asian 

3.8 

6.2 

Other 

3.8 

4.1 

Free-lunch eligible 

65.8 

67.4 

Limited English Proficient (LEP) 

2.5 

1.7 

Special Education 

1.9 

0.8 


Mean 

Mean 


(SD) 

(SD) 

Grade 8 pre-test math scale score 

363.82 

363.45 


(6.83) 

(7.22) 

Grade 8 pre-test reading scale score 

360.05 

359.77 


(7.10) 

(6.65) 
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